DOCUMENT RESUME 

ED 305 928 IR 013 797 



AUTHOR Mandinach, ^ Ellen B.; And Others 

TITLE The Impact *of the Systems Thinking Approach on 

Teaching and Learning Activities. 

INSTITUTION Educational Testing Service, Princeton, N.J. 

REPORT NO ETC-TR^88~31 

PUB DATE Dec 88 

NOTE 102p. 

PUB TYPE Reports - Research/Technical (143) 



EDRS PRICE MF01/PC05 Plus Postage. 

DESCRIPTORS ^Cognitive Processes; ^Computer Simulation; 

^Curriculum Development; Hiqh Schools; History 
Instruction; * Instructional effectiveness; Learning 
Activities; Problem Solving; Psychological Studies; 
Science Instruction; ^Systems Approach 

IDENTIFIERS ^Systems Thinking and Curriculum Innovation Pro^ 



ABSTRACT 

The Systems Thinking and Curriculum Innovation 
(STACI) project is a multi-year research effort intended to examine 
the cognitive demands and consequences of learning from a systems 
thinking approach to instruction and from using simulation-modeling 
software. This study tests the potentials and effects of integrating 
the systems approach into science and history courses to teach 
content knowledge as well as general problem solving skills. Also 
examined is the effectiveness of STELLA, a simulation-modeling 
software program, as a tool by which to examine scientific and 
historical phenomena. The focus of the research is on the learning 
outcomes and modeling- performance that result from introducing an 
instructional environment that enables students to learn from and 
make concrete multiple representations of dynamic phenomena. Analyses 
of the data are presented in 14 tables and 12 appendixes include 
additional data, worksheets for various tasks, and a description of 
ways in which the systems thinking approach was integrated into three 
science courses. (16 references) (Author/EW) 
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The Inpact of the Systens Blinking i^roacii 
on Teaching and Learning Activities 



Ihe Systems Thinking and CurriculuiP Innovation (STACI) 
Project is a multi-year research effort intended to examine 
the cognitive demands and consequences of learning from a 
systems thinking c^proach to instnx^ion and frcm using 
sijmilation-^nodeling software. The purpose of the stucJy is ti^ 
test the potenti2as arri effects of integrating the systems 
approach into science and history courses to teach content 
kncwledge as well as general problem solving skills, ihe 
study also examines the effectiveness of using STELLA, a 
sinoOatiGn-modeling software progr a m, as a tool by vMch to 
examine scientific and liistorlcal phencniena. The research 
focuses on the learning outccmes and modeling performance 
that result frcm introducing an instnictioned envircaiment 
that enables students to learn frcm and mate concrete 
i milti p le representations of dynamic phenomena. 

General Background Information 

The intent of the Systems Thinking and Curriculum Innovation 

(SIRCL) Project is to examine the teaching and learning activities 

that result from introducing a systians thinking abroach to 

instruction in seconcJary school science and history. As defined 

here, the systems thinking approach consists of three individual 

tut interdependent conpcnents. First, there is system dynamics, 

the theory on vAiich the instrtictionad per^ssctive is based. The 

second oonponent is STEUA (Structural Thinking E>q)eriroental 

Learning laboratory with Animation; Richmond, 1985; Richmond & 

Vescuso, 1986) , a software package that can be used as a tool to 

teach systems thinking, content knowledge, and problem solving. 

The third ccnponent is the Macintosh microcorputer on vAiich the 

STELEA software runs. The research focuses on the curriculum 

development that integrates the systems approach into existing 
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courses and the learning outxxanes that result from using such an 
approach and software in classrtxn settings. 

The STAd Project, v*iicJ; began during the 1986-1937 acadeitdc 
year, was a two-year research effort now concluding its final year. 
Ihe study was conducted at Brattleboro Union Hi^ ScJiool (BUHS) , 
Brattleboro, Vennont in v*iich four teachers used systems thinking 
in their courses. Content areas included generEd physical science 
(GPS) , biology, chemistry, physics, and an experimentad history 
course entitled War and Revolution. The purpose of the project was 
to examine the extent to which students acquired higher-order 
cognitive skills through ej^xDsure to and interaction with curricula 
infused with systans think^Jig and subsequently generalized 
knowledge and skills to tasks in other substantive areas. 
C3cnparisons were drawn between traditionally tauc^t courses and 
those that used the ^^stans approadi. Ihe research enabled the 
examination of skill and knowledge transfer across content areas as 
students were exposed to courses that used the systems approach. 

Three ancillary studies were conducted in oonjunctiai with a 
main classroom study and reported elsedftiere. The first substudy 
(Mandinach, 1988b) focused on a select qcojg of students \it}o 
received extensive exposure to the systems approach in the War and 
Revolution seminar. These students were studied in an intensive 
case study format. The objective of this stu^ was to collect 
indqjth information about the students^ thou^t processes, 
performance patterns, knowledge, and general problem solving 
skills. The second substudy (Cline, 1988) examined the 
organizational iiK|>act of the introduction and inpleanentation of 
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systesns thiridng in the hic^ sctool. Ihe objective was to analyze 
changes that occur in the strvicture and functioning of the school 
as a result of the curriculum innovation. Uie third substudy 
(Mandinadi, 1988a) was a clinical cases stucty and cognitive 
analysis of students' perfomanoe and aoquisitiOTi of hi^er-order 
thinking skills on a subsanple of chemistiy and physics students. 

The purpose of this document is to r^rt on the curriculum 
development, teaching activities, and learning outocmes that 
resulted in tWD years from the integration of the systems a^^proach 
into several hi^ school science and history courses. We provide 
descriptions of systems thinking, the site, the design, data 
collection, instrumentation, curricula (^pendix M) , and cognitive 
outocmes frcm the main dassrocm study. 

The Systems Thinkincr Approach 

Systems Ihinkinq 

Systems thinking is a scientific anedysis technique that 
provides a means fay vMch to understand the behavior of oonplex 
phenomena over time. In recent years s^preciation has developed 
particularly for the heuristic value of systems thinking. The 
creation and manipulation of models is increasingly recognized as a 
potentially powerful teaching technique. Based on the oonoept of 
change, system dynamics uses simulations and ocnputer-based 
mathematical models to represent ocnplex relationships among 
variables in the OTvironment (Forrester, 1968) . It is possible to 
understand the rule-like behavior of systems by constructing models 
of variables and their interactions, and examining the cause-and- 
effect relationships among the variables. Ihe notion of a system 



is based on: (a) variables that diaracterize a system and change 
ever time; (b) ralationshiis anoref variables are interconnected by 
cause-and-effect feedback loops; and (c) the status of <xie or more 
variables subsequently affects the status of other variables. 

Simulation models, sinplified representaticxis of systems over 
hypothetic2d time, are used to examine the structure of systems. 
Using simulations, characteristics of selected variables can be 
altered and their effects on other variables and the entire system 
assessed. To build a simulation, it is necessary to understand the 
major variables that ccnprise the systan. Ihese variables are used 
to form dynamic feedback systems, expressed in simultaneous 
equations. Over time, variables change and subsequently cause 
other variables and their interactions to change. Thus, systens 
focuses on the connections among the elements of tlie system and 
provides a means to understand how the elements OOTrtxibute to the 
v*Dle (Roberts, Andersen, Deal, Caret, & Shaffer, 1983). 
STEHA and the Macintosh 

Bie concepts that underlie the field of system dynamics form 
the basis for much of the simulation software that currently is 
used in educational settings. Uhtil recently, the instnK±ional 
lase of systaBB thinking was oonstrziined to environments that had 
pcMerful mednftame ccnputers. The advent of a new software 
product, STELtA, has made it possible to operaticxialize these 
concepts on a microoonputer in a user^firiendly environment (n.b, , 
ricxod/namo has b&m available but language constraints make it 
particularly cunberscne and difficult to use) • SIELlA csqpitalizes 
on the gra{£iics and loon technology of the Macintosh microccnpiter 



(several vdncJcws — striictural diagrams, equations, graph pads, 
tables— are available to the learner) , ther^ enabling individuals 
not versed in the intricacies of mathematical modeling to create 
their cwn systems. By minimizing the mathematiccd and tedinical 
sjcills needed to construct models, STEIIA facilitates the creation 
and manipulation of ocnplex models of system phenomena. 

STEIIA facilitates student introductions to arcaytic prxDblem 
solving perspectives inhereait in ssystems thinking throu^ an 
iterative process of sinulation model cxanstniction. Model-txiilding 
requires learners to formulate, test, and revise Iqpotheses about 
relations within the dynamic systems. Modeling, as conceptualized 
in STEIIA, is a three-step process. First, learners use STEUA's 
"tool kit" to create diagrams rqsresentative of the systems to be 
modeled. These diagrams a^based on relational assunptions and 
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Icgic hj^xathesized by the users. As learners cxeate stnxturcil 
diagrams, SEETIA translates them into sets of equations that 
represent the systems. Second, learners then fomally specify the 
logic that ocnnects the parts of the systems. STEUA's graphics 
capabilities facilitate this logical translation by providing 
visual of the connections among the ccnponents of the systems. 
Hie simultaneous equations are brised on the mathematical 
assunptions and values learners si^jply to define the variables and 
the relationships among them. Finally, STEIIA dynamically runs the 
systems as simulations over hypothetical time, given the logiced 
assunptions provided by the learners. Results of the sinulation 
stimulate revisions, thus creating em iterative sequence of 



fornulate-test-revise st^ that lead to the developnent of dynamic 
models of phencnena over time. 

Site Descriptiai; Brattl^ro IMion Hic^ School 
BOHS serves a rural five-tcwn district in southeastern Vermont 
v*iose population is approximately 20,000. The sc1kx)1 has nxqhly 
1,600 students and a faculty of 80 teachers. Since 1985, BUHS has 
been the site of a nunber of systems thinking activities, all with 
the purpose of introducing students, educators, awi the public to 
the principles that imderlie the field (Ifendinach & Bwrpe, 1987) . 

Four teachers formed the systems grocp at BUHS. All were 
trained to use the systans appxxxidi and integrated this perspective 
into their ocurses. One course, entitled W&r and Revolution, was 
heavily infused with systems thinking and the use of SIEEIA. 
Systems thinking formed the basis for this course. In contrast, an 
integrative approach was used in the scieioe courses. The a^aproach 
was integrative in that the teachers identified oonoepts within 
their cxirricula that could be enhanced by the use of systens 
princ^les. Bather than teadi particular concepts as they had in 
the past, the systans teachers explicitly en|hasized the systemic 
nature of the topics, noting sucii ideas as causality, feedback, 
variation, and interaction. Ihe courses covered the same body of 
knowledge tau^t in the traditional curriculum, but specific 
oonoepts were disnisspd from a systems per^)ective. Ihese courses 
will be described in greater detail in the curriculum section. 
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Desicgi and Data Oollection 

Design 

In the 1986-1987 academic year, systems think 'ng was 
integrated into three GPS, four biology, and three chemistry 
classes (Table 1) . An equivalent number of traditional (control) 
ccurses v^ere tau^t oorcurrently by other menbers of the faculty. 
In the project's second year, the systems e^roadi was lased in two 
GPS, four biology, three chemistry, and three pities classes, ihe 
traditional treatment contained one additional class per subject, 
but no physics classes^ 
Data Oollection: Instrumentation 

Severed t^pes of instrumaits were used to assess ability, 
oontait-q)ecif ic knowledge, systems thinking, and hi^^ier-order 
thinking skills in various stages of the research. Ihese 
instruments included pretest, in-dass topic, and posttest 
measures. General information about the instruments can be found 
in Appendix A. 

Pretests were used to assess foibjects' ability and crntent- 
^5ecific knowledge. BUHS si^plied the students' most recent 
standardized aciiievement test scores. The California Achievement 
Tests served as roug(h estimates of general ability. EES also 
administered a sanall battery of tests, including the Advanced 
Progressive Matrices (Raven, 1958, 1962), to provide another index 
of general ability, other measures ^ated to skills hypothesized 
as iii|x>rtant concepts underlying systems thinking were given. 
These included inductive and deductive reasoning, f igural 
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anedcsgies, and understanding relaticansh^s and were parallel fonts 
to the tests administered in the project's first year. 

Modified versions of previous finad examinations were given 
to both the systems and traditional classes. The GPS, biology, 
chemj.stry, and physics teachers took last year's tests, identified 
critical, yet basic conc^sts, and gave the shortened versions to 
their classes early in the academic year. Ihese tests served as 
baseline assessments of content knowledge in the subject areas. 

Teachers gave content tests and exercises in their courses 
during the year. Teachers attenpted to make these activities 
cooparable for the ^stems and control classes in their subject 
coverage. In GPS, the systems and traditional teacher collaborated 
cn a test of Speed and Motion. In biology, the systens teacher and 
the second traditional teacher collaborated on transport and 
imnmology tests. 11 ^ ocninon chemistry test was on rates of 
reaction. Each systems teacher also prepared an open-ended problem 
that related to science content covered hy systems modules. These 
probleans were given to both systems and traditional classes. 

Ihe tead-jers also asked teachers to prepare and arhnipitetor 
ooDiuon fined examinations to their classes so that v?g would be able 
It' cmpare differences in content knowledge that resulted frtan 
using the systems approach. Posttests were given by all the 
systeans teachers and the traditional GPS teacher, ttifortunately, 
the traditionetL chemistry and biology teachers failed to collect 
these critical data. However, frm the end-of-year test given, v/e 
were able to natdi in content covereige, enough biology items across 
teachers feon v*iich to loake conparlscns. 



EES developed a instrument that was used to acyy^^s 

kncwledge of systaws thinking and SIEEIA. The instrument contained 
items of increasing difficulty that measured a broad ^jectnim of 
skills along a continuum ranging frcm elementary conc^yts to 
ccnplex modeling skills. Ihe 1987-1988 versicn of the Systems 
Biinking Instrument (STT) was a revision of the test piloted last 
year. It was based on iiput frcm the BUHS teacSiers and systens 
o}q)erts and through rational task analyses. Measures of systens 
thinking focused on ooncxpts such as knowj xJge of gre^ahing, 
variation and vari^les, causation and causality, feedback, looping 
constructs, modeling, and smiA. This test was administered at 
the end of the year to only the systems thinking classes. 
Data Oollection; Ctoservations and Interviews 

Classroom observations were conducted during a number of site 
visits during the two-year project. Three project niesnbers observed 
both systems and traditional: classes to obtain information about 
course content, structure, and classroom procedures. Systene 
classes were observed ^fihai systems modules as well as traditional 
materials were presented. Observations were scheduled v*ien similar 
topics were covered to see how the systems and traditional teachers 
differed in their approach to the hi^i^ted cono^>ts. For 
exanple, we observed hew the chemistry teacners presented the topic 
cf reaction rates, noting differences in emphases, presentation, 
and other areas due to the use of systens concqpts. 

Interviews with systems and traditional teachers were 
oottJucted to obtadn aviditional infonr^tion about the classes. It 
was critical to gather information frcm the systems teachers 
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oonoeming the i s s aip s they cxxifronted during the iiiplementatic»i of 
tl>e curriculum innovation. The interviews also provided an 
cppnrtunity to proiDe teachers about their pero^jticais of the 
systems thinking modules, iirpleanentation difficulties, and other 
issues related to the effects of the curriculum innovatiai on their 
teaching activities. 

To examine variation in content eoophasis, all science teachers 
viere asked to provide information on the amount of coverage of 
different curricaxltmi topics. T^ie systems teachers edso were asked 
to indicate the time devoted to instnx±ion in systens thinking and 
to vdiich topics the approach was e^lied. 

Results 

Discussions of the results will focus on four t^pes of data. 
First, ability test performance is reported. Ihese data include 
the California Achievement Test (CAT) and the Advanced Progressive 
Matrices (Raven, 1958, 1962) whidi served as laaasures of general 
ability. Other tests fron the reference battery served as measures 
of skills l^pothesized to be related to systems thinking. Third, 
performanoe on the STI is reported. A ccnplete description of the 
rationale for and content of this test is discussed. Finally, 
performanoe on content tests in GPS, biology, ctenistry, and 
physics is doognented. A discussion of the relationship of these 
tests to the systems thinking curricula also is provided. 

The various analyses reported here require examination of 
different parts of the sanple. Oonsequently, sample sizes differ 
in accord with the measures used in the ansdyses. TVo caveats 
should be noted. First, because of ooncurratt enrollment in more 
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than one systenas class, special treatanent of the data was necessary 
to account for overla(]piiig courses. Thus, scores are sometimes 
rqporteci for the science courses without physics. Second, it 
should be no'ced that students in the War ani Revolution SCTiinar are 
i n cluded in sane of the analyses to provide a ccnpariijon grxx^) 
(particularly for die sn) . However, they are not inclixJed in the 
content test analyses, vAiich focus only on the science classes. A 
detailed descr^]tion of perf onnanoe in the seminar can be found 
else^Aiere (Mandinach, 1988b). 
Ability and Acfaievanent Measures 

IVro luaasures of ability were used to g^-^ jge students' general 
inteL..«A±ive functioning. First, students' most recwt 
standardized achievanent test scores were used to as ses s general 
crystallized ability (Cattell, 1971) . Crystallized ability refers 
to previously constnacted asseoblies of perfonanoe processes 
retrieved as a s^sbun and-applied anew in familiar instnxrtJLonal 
situations (Snow, 1980, 1982) . Biis constnict reflects, icrxj-tcrm 
accunulation and organization of knowledge and skills. Ihe 
reading, language^ and mathematics subscales and total test score 
from the CM were extracted frtm students' sohool records. To 
ehhanoe interpiretaDility, national percentiles are repeated i;ere. 

Second, JParts I and II of the Advanced Progressive Matrices 
(AIM) were used to assess general fluid ability. Fluid ability 
refers to new asseooblies cvtaptai to new and unfamiliar situations 
and novel problems by reassenbly of available performance processes 
(Snow, 1980, 1982) . Par the 1987-1988 administration of the 
matrices, the six even item fitxm Part I and the first nine even 
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itans f rem Part n were given to the students as part of the 
cognitive reference battery. Ihe reliability for the instnirnent, 
vsiTJ^ the Speaman-BrcMi fonnula for imequal lengths, was 
calculated at r - .90. 

average, students in science (M = 12.11, S.D. = 2.88) and 
physics (M = 13.06, S.D. = 2.22) courses performed well on the AIM 
(Table 2 and Appaidix B) . Students seemingly performed better in 
the more advanced courses with War and Revolution students soorii^ 
the hi^2St (M = 14.70) and those in GPS the lowest (M = 9.92) . 
Biology (M » 12.12) and chaaiistry (M « 12.66) students scored 
between the extremes. Furthermore, students in the traditional (M 
= 12.15) and systems (M = 12.08) classes performed conparably. 

Data from the CAT were available for most of the BUHS 
students. BUHS sci^ioe students averaged in the 73.31 percentile 
in reediixf, 68.68 in language, 73.75 in mathematics, and 72.66 on 
the total test. Apppniix C pfcesents the results of the achievement 
tests for the systans thinkincr and traditional classes In biology, 
chemistry, ptr/sics, and War and Revolution. GTB studerH-? scored 
lowest on the CAT. The data also indicated that cne students in 
War and Revolution were outliers, ccnpared to others in the sample 
as well as nationally. 

Statistical ^analyses were performed to determine if there were 
pre-existing differences in aciiievenient test scores between the 
traditional and systans classes within subjects. Differences were 
found betwem treatments for GPS on the ca^ scales. Ihe 
traditioned classes were »re able than the sfystens classes 
(reading F(l, 57) - 2.68, ns; language, F(l, 57) ^ 8.51, g < .01; 
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inathenatics, F(l, 57) = 9.22, E < .01; and total test F(l, 57) 
8.15, E < .01) . A differenae on the math subscale was noted in the 
biology classes, F(2, 147) = 4.28, g < .05. This difference 
occurred as the systems classes performed better than those tau^t 
by traditional teacher 1 (T{92.8) = 2.86, g < .05). No differences 
v^ere fcxmd between £»ystesns and traditional chesnistry classes. 
Rsferenoe Battery 

Four other tests were administered with the AEM in a reference 
battery. The intent was to assess students^ performance on skills 
related to sfystens thinking* The four tests were the Figural 
Analogies (lA) , Diagranming Relation^iips (CR) and Letter Sets 
(IS) (Ekstrom, French, and Barman, 1976) , and Deductive Reasonirer 
(CRT) . All hut the CRT were parallel forms of the versions 
administered in the previous academic year. Results between the 
Year 1 and Year 2 tests are presented in Table 3. rositive 
correlations were found betue^ the two edministrations. The 
magnitude of these relationsh^ were modest, but significant. 

Figural Analogies . FA (Apoendix D) consisted of 10 itents 
presented in a four-alternative, multiple-choioe fonnat. A figural 
analogy item consisted of a typical analogy item {A:B: :C:D) , but 
used geanetric figures, such as triangles, squares, circles, etc. , 
rather than ^iKsrds^ Mean performance on the items ranged from .20 
to .94, with a split-*alf reliability of r = .54. One item 
appeared to present particular difficulty (see Appendix E) . 

Performance differences were noted on the EA. The systems (M 
= 7.06) and traditional (M = 7.19) groups performed conparably. 
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Stuc3ents in physics (M = 7.42) scored the highest, followed by 
chemistry (M » 7.41) , biology (M « 7.08) , then GPS rM = 5.70) . 

Diaqrairitiincf Relationships . OR (i^spendix F) consisted of 15 
items presented in a five-alternative, multiple-dioice format. The 
purpose of the test was to see if students were able to discern ard 
diagram relationships among grcx^ of things. For example, a 
sanple prcblem asks stuclents to determine the relationships anraig 
birds, pets, and trees. Hie correct response would indicate that 
trees are neither pets nor birds, but seme birds are pets. Ihus, 
ther> should be an interse c tion between pets and birds, but not one 
betwe^ trees and pets or birds. 

Performanoe on the itans ranged fixm .34 to .93, with a mean 
score of 9.97. Ihere were 10 items on vAiich the p values were 
under .70 (i^^)endix G) . Bie task was difficult and imlike any the 
students had encountered previously. The split-half reliability 
(Spearman-BrcMi for unequal parts) viis r - .82. 

A similar pattern of performance differences was noted on the 
ER. Ihe systenti (M « 9.89) and traditional (M = 9.90) 
perfonned ocfir.?e.^^..Y Studaits in physics (M = 11.60) scored the 
highest, foU;^^.^ ry c^Toistry (M = 9.88), biology (M = 9.85), then 
GPS (M « 7.72*.. 

Letter Se^;s . IS (Appendix H) also consisted of 15 itens 
presented in a five-alternative, multiple-choice fonnat. Each item 
oontzdned five sets of of letters in grot?js of four. A ocmnon rule 
linked four of the five letter sets. IS required identification of 
a nile that made four sets of letters similar in sane way, and one 
set different fron the others. For exaoople, NDPQ CEFL ABCD HUK 
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UVWX, is one of the sanple itens. ihe rule here is fcxir letters in 
alphabetical sequence. Ihe seoand set violates that rule, and thus 
is the corxBct response. 

Perfonnanoe on the items raixfed from .08 to .92, with an 
average total score of 10.21o Perfonnanoe on this task was more 
consistent. Hcwever, the p values indicate that stojidents may have 
had difficulty finishing the task within the time constraints. 
Perfonnanoe on Itans 28, 29, and 30 reflect this possibility (see 
Appendix I) . The ^it-half reliability (S^jeannan-Brown for 
xmeguaa parts) was r = .86. 

Again, a similar pattern of perforxnaiioe differences was noted 
on the IS. The systems (M « 10.21) and traditional (M = 10.22) 
grtxps perfanned ccnparably. Students in phys' (M = 10.77) and 
chanistry (M = 10.68) scared the highest, followed by biology (M = 
9.94) and GPS (M » 8.67) . 

Deductive Rsasoninc? Test . Ihe final test, CRT (i^apendix J) , 
recyiired students to dgduce the relationships among information 
given in a verbal problem. Ihe mayimim score was 15. Students 
averaged 6.96 points. The systaoos (M = 7.42) students performed 
better than those in the traditional (M = 6.39) classes. Chemistry 
(M « 8.77) and physics (M « 8.56) students scored the highest, 
followed by biology (M = 5.54) and GPS (M = 3.64) . The systems 
chemistry classes (M « 10.45) performed well above all other 
classes. 

Relationship Amoncr the Ability and Referenoe Battery Measures 

Table 4 presents the interoorrelations among the COT, AFM, and 
the reference battery measures. Ihe correlations among the 

16 

[7 



reference battery measures were found to be moderate^ but 
significant, similau: correlations were found between the reference 
battery and the achievement test scores. CAT total score and APM, 
the two primary indices of general ability^ showed a mcxierate 
correlation of r(341) = .46. InteroorrelatiOTS between the 
subscales of the CfO! were extremely strong. 
Systgns Thinking Instrument (STI) 

This section focuses cn the prixoary instrument by vAiich 
knowledge aoguisition in systans thinking was assessed. Ihe 
rationale for and oonbant of the instrument will be described 
first. Results then will be presented^ with oorparisOTS itade with 
the first administration. 

Test characteristics . The sn (J^ppendix K) consisted of 68 
itans that were intended to assess a rangfe; of skills thou^ to 
underlie systans thinking. A rational task analysis^ in 
conjunction with revisions fixm last year-<see Mandinach & Ihorpe, 
1988) , yielded eic^t skills which then were made into subscales of 
varying lengths. Table 5 presents the alpha reliabilities for 
those scales, ihcj 68 itans on the total test yielded an alpha of 
.95^ indicating that the test was extremely consistent across itens 
and subscEtLes. 

KWf/ledge of basic grs^jhing concepts (e.g.^ labelir^ and 
scaling axes^ coordinates) ccnprised the first subscale. The five 
gra^jhing itans yielded and alpha of .45. This was the least 
intemedly consistent scale. T!^ other sccdes focused cn graphing 
skills. A first required interpretation of grs^^hs. Students were 
asked to interpret a grs^ and provide a verbal description. The 

17 



seven items on this scale yielded a reliability of .60. The second 
siibscale required translation. Ihat is, students ^ere asked to 
take a verbal description of a problan and translate it into a 
graphical r^artsentaticxi. Ihere were six such iteanis (o(.= -62) . 

One other scale focused on mathematical skills that relate to 
systems. This scale focused on students^ understanding of 
functions. Students were asked to write functions, find slopes, 
and grs^ functions. Ihe three items yielded an alpha of .84. 

Two scales were designed to assess knowledge of concepts 
critical in systems thinking. The first focused on causality («< = 
.94) . Here students were asked to ocnplete a causal relationship 
and a caused diagram. Defining caussdity leads directly into the 
concept of looping. Four items = .82) required that students 
either interpret or construct a causal loop diagram. 

The fined two scsdes measured skills and knowledge unique to 
systems, STEIIA, and modeling. The first scale cor^isted of sisple 
identification items. One set of identifications required students 
to determine if a variable was a stock (level) or a flow (rate) , 
th^ define its unit of measure. A second set of items asked 
students to identify parts of a structural diagram (e.g. , stock, 
connector, flew, converter) . The 33 items yielded an alpha of .95. 
The final sccde focused on the construction and interpretation of 
systems m odel s of varying ccnplexity. There were seven itens in 
the scale which yielded an alpha of .79. At one end of the 
continuum, studaits were asked to take a simple model and identil^ 
hew certain variables eiffect other variables. At the other end of 



ERIC 



18 

I ^ 



the cxjntiruum, students were asked to take a verbal descaripticai of 
a paxblen, oonstmct a model, and then interpret that model. 

There was a rationale for oonstnictixig such a diverse test. 
First, we wanted to examine hew the items perfonuBd and how they 
related to vAiat the students had encountered in class. Wfe nee„^< 
to construct an instrument that would be both reliable and valid 
with respect to dieir classroom eoqserienoes with systeas thinking. 
Perf onnanoe on such an test would inform EES and the BUHS teachers 
in terms of studeaits' learning outocraes and cognitive prxx»ssing. 

Second, and pertie^ most iaportantly, we wanted the sn to be 
reflective of and sensitive to differences in the systens curricula 
across the subject areas. As docunaited in a previous r^rt 
(Mandinach & Thorpe, 1987) , each of the teachers introduced systems 
conoq>ts to their classes differently. That is, they focused on 
particular oonoqjts of varying difficulty and integrated them into 
their courses with different degrees of intensity. Thus, the 
instrument should yield ranges of perf ormanoe within the courses 
around conceits that were either stressed or briefly described. 
The sn was constructed to provide information about these ranges 
that would be reflective of the curricular differences. 

Interrelationships among the subscales . T&ble 6 presents the 
correlations among the STI's scales. All correlations were found 
to be sigiiificant at the g < .01 level. The gre^iiing scale showed 
moderate correlations with edl other scales, whereas interpretation 
and translation of graphs were most strongly related to other 
scales. In particular, interpretation and loops (r(212) = .48), 
translation and identifications (r(212) = .50) , and loops and 
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identifications (r(212) = .46) were most highly related. 
Understanding the basic concepts of systens thinking (the Systens 
Identification scale) was strongly related to performance on the 
systems thinking scale (r(212) = .74). 

Te£.t performanoe . Item-level performance data are presented 
in ;^paidix vAiich is is broken down by subject area in order to 
examine the progression of skills and knowledge across courses. 
Table 7 presents the descriptive statistics for the subscales. 
These data are further brbkeai down by course in T&bles 8 and 9. 
Wiereas data in Tables 7 and 9 are presented as raw scores, percent 
correct is reported in Table 8 to serve as a ccnparisGn between the 
test's two administrations. Data discussed in the text also are 
reported in terms of percent correct to aid interpretability. 

On all three of the graphing subscales (Gra^^hing, Gr^iiing 
Interpretaticn, and Grs^jhing Translation) performanoe increased in 
the more advanced courses. Students in GPS scored the lowest, 
follcwed by biology and chemistry, ihe physics and War and 
Revolution students performed oonparably. Interesting to note are 
the differences between Years 1 and 2. Whereas exti^mely hi^ 
scores were found in Yeeu: 1, performanoe in Year 2 was 
substantially Icwer (Mandinach, 1988b) . Chemistry performanoe was 
consistent across the administrations, vAiereas substantial 
improvements were noted for the GPS and biology classes. 

Indepth analyses indicated particular problem areas 
experienced by the students. Consistent with last year, the GES 
students did not understand the (x, y) convention of defining 
points within a coordinate system. Ihis deficit led to poor 
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perf onnancse on itans that reqirb^ graphing points and defining 
axes. It eOso led to cjonfusions vAen they were asked to translate 
or interpret graphical problesojs. 

GPS students continued to show no understanding of functiois. 
Ihey were unable to graph functions, determine slopes, or vorite a 
function v*en given a graphical represertation. The GPS students 
achieved only 4 percent correct on this subscale, in contrast to 26 
percent for biology, 42 penult for chanistry, 64 percent for 
physics, and 82 percent for War and Revolution. Functions were a 
difficult concept to understand and apply in gre^iiic form. 

Interesting performance patterns emerged on the two scales 
related to variation and causality (Causality and loops) . For both 
scales, the War and Revolution students achieved the hi^iest 
scores, followed by those in biology, physics, chemistry, then GPS. 
Ihe fact that the biology students scored hic^ier than the other 
science classes on these scales is reflective of the "instructional 
eniliases stressed by the biology teacher. The itens in the 
Causality scale required students to ocnplete a causal statanent 

(e.g.. Amount of studying causes .) , then make the caus€a 

locp. Last year, GPS students coopleted the statements, but had 
particular prcblans with the siaple causal loops, obtaining cnly 31 
percent correct. Hiis is in con t rast to 58 percent for biology, 65 
percent for chanistry, and 98 peroort for War and Revolution. 
Performance in the second year was substantially better, with GPS 
achieving 72 percent correct, 92 percent for biology, 86 for 
chemistry, 91 for physics, and 100 for War and Revolution. 
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Oonpletion of more cscnplex causad loops were required in the 
Loops scale. Berfonnance was lower but aralogous to the Causedity 
scale (I53PS = -SS'' Ifeiology -73; I5chemistry -60; Mphysics 
.67; and fj^^R = .74) . Hie GPS students €)q)erienoed particular 
difficulty in translating a sinple verbal description into a loop 
diagram. The GPS teacher edmittedly devoted a limited amount of 
time to loops. Uius^ we interpret these results to reflect the 
smedl amount of esqxssure the GPS students were given to loops. It 
is noteMorthy that on this more oonplex task, bi ^logy students 
(sophcmores) did as well as War and Revolution students (seniors of 
exceptional ability) . 

tti the more elementary scale that tested basic knowledge of 
systems thinking and SIELIA, the Systaos Identifications scale, 
there were several notable trends. First, the War and Revolutic»i 
class (M - .81) achieved the highest score, yet at a substantial 
decrement from last year. These studaits were not able to^dentify 
inflows, outflows, or stociks, the basic eleownts of a systens 
model. In contrast, such identifications were routine for Year l 
fipi nin a r students. These differences reflect the teacber^s 
iii{>lem^itation of systans in the two years (Mandinach, 1988b) . 

GPS students (M » .28) also eadiibited weaker performance in 
Year 2 (M = .42) . In contrast to the War and Revolution seminar, 
GPS students identified parts of a mcxJel, but had trouble with 
units of measure. Performance was equivalent for biology and 
chemistry (M = .63) and slic^tly higher for pt^ics (M = .72) . 
Both chemistry and biology performance improved from Year 1 to 2. 
Results indicated that the biology, chemistry, and physics students 
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perfonned better on the structural nodelijig identifications and 
were less adept at the units of measure and variable itens. 

A similar performanoe pattern was found for the Systems 
Uiinking scale, vAiere the focus was on nodel interpretation and 
mocJeling. Given that the GPS students had little exposure to these 
ocnplex ootio^jts, they ac±deved only 26 percent correct. Ihey at 
least made seme attenpt to interpret the models given to them, but 
had slighUy more difficulty constructing their own models. The 
biology (H = .43), chemistry (M = .42), and physics (H = .46) 
classes again performed similarly, viiereas the War and Revolution 
seminar perfonnad only slighUy better (M = .55) . Inproved 
performanoe was noted in GPS, biology, and chemistry. However, a 
substantial decremaTt was observed in War and Revolution. In Year 
1, seminar students achieved 86 pertjent carrect, indicating their 
ability to construct ocnplex stnictural models. Year 2's seminar 
students, de^ite the extensive expoBuve to systens thinking,-were 
unable to evidence nodaling skills. Again, this is reflective oi 
changes in the course (Mandinach, 1988b) . 

The contrast between Year 1 and Year 2 performance is striJdng 
(Mandinach & Thorpe, 1988) . last year, every item on the subscale 
e^diibited the same performance trend across courses (i^pendix G) . 
War and Revolution either achieved or e^proached 4 perfect score. 
These scores were well above the science classes. GPS students 
adiieved the lowest scores, vAiereas the biology and chanistry 
performed similarly. Year 2 performance was substantially 
different. GPS again scored the lowest on all items. However, War 
and Revolution perfonned poo*:ly in ocoparison with the first class, 
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and only slighUy bet±er than Year 2's science classes. More 
distressing is the relative performance on Iten 28B, the ocnplex 
modeling problem. Not only did the physics students score better 
than those in the seminar, but chemistry students performed 
esctremely poorly, it is clear that whereas the physics and biology 
teachers gave attention to modeling concepts, such enphasis was 
lacking in War and Revolution and chemistry. 
Oontent Tests 

A nunber of content tests were eKSministered in the science 
classes to examine the iapact of systens thinking as well as to 
as s es s typical instruction. 

In GPS, both the traditional and systems teachers 
collaborated to give 40-item pre- and posttests of knowledge of 
physical science. They also prepared a 30-item test on speed and 
motion (SSM) , the topic of the primary systems module. Ere- and 
posttest performance were moderately correlated (r(34) = .34, 
.05) . me pretest showed sli^tly higher correlations with S«M 
(r(33) = .38, E < .01) and Sn (r(31) = .36, g < .05). In 
contrast, the posttest was highly related to performance on both 
S«M (r(33) = .62, g < .001) and the STI (r(32) = .72, g < .001). 
Students \itio viiere successful on the S«M test also performed well on 
the sn (r(32) = .72, g < .001). 

Traditional students = 16.72) performed better than those 
in the systems classes (M « 14.00) on the pretest (F(l,46) = 8.44, 
E < .01) . No differences were found en the posttest or SiM, 
However, gadn score from pre- to posttesting e^rraached 
significanor^ (F(l,46) « 3.27, g - .07), indicating that the systans 
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caasses (M = 6.56) made substantial progress in cxxiparison to the 
traditional stidents (M = 4.24) . 

Further analysis of performance indicated that systems and 
traditiaial students differed in their answer and error patterns on 
the S&M test. Uie traditional students excelled in their precise 
use of terminology, ihey were able to identify and define patterns 
of speed and motion on simple graphs. Although these students were 
able to recognize oarcegts in isiaple formats, they were unable to 
identify those same ccno^^ts in mace oanplex prcblents. In 
ocxttrast, the systems students were xle^t at deoonposing parts ' -o 
understand entire problems. The systems students did not 
CJonoentrate on ter m i n ology. Instead, they provided descriptions of 
bdiavior over time, rather than labels for oonoepts. 

Biology. In biology, 44-item pre- and posttests were given to 
assess knowledge of biological oonoqjts. 'Ihe pretest was given by 
all three teachers, ihe systems teacher ettlministered the entire 
posttest, vhereas parts of the finzd measure were given by the two 
traditional teachers. B:js, the samples differ, based on the 
content of test admi n istration. Ihe systems teacher collaborated 
vdth the second traditional teacher on an 11-item test to assess 
tranqxart and a 31-item test to measure innunclogy. Itaurteen 
selected icems from the innunology test that relate, to causadity 
were separated for further examination. 

Scores on the biology pretest were only moderately related to 
posttest perfc^manoe (r(72) « .23, g < .05), Indicating that those 
v4io performed well at the end of the course were not necessarily 
the same studarts vho had done well earlier. The pretest was not 
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related to perfonnanoe on the innunology (r(63) = ,11) , selected 
imounology itons (r(63) = .09), or transport (r(43) = .14) tests. 
In contrast, perfonnanoe on the posttest was related to performance 
on the iitraunology (r(63) = .51, £ < .001), selected ijiinunology 
itenjs (r(63) = .40, g < .001), and transport (r(48) = .52, g < 
.001) tests. Students v*o performed well on the tran^rt test 
also did well on the innunology test (r(45) = .58, 2 < -001) and 
selected innunology itene (r(45) = .58, g < .001) . 

Exaninaticn of the tliree biology teachers' classes' posttest 
perfonnaxioe indicated that the systems classes (M = 4.96) scored 
between the two traditicml teachers' clas':<is (Mn = 3.60; 
6.53) (F(2,155) = 32.74, £ < .001). Overall when t^e two 
traditional teachers' results were ocnbi' 3d, no differences were 
famd between performance by the systens (M = 4.96) and traditicsial 
classes (M = 4.76) (F(l,156) - .37, ns) . 

ilie systems and second traditional teacher gave tests of — 
kncwledge of innunology (F(l,92) = 12.47, £ < .001) and selected 
innunology items (F(l,92) = 6.69, £ < .05) on which the traditional 
classes performed better than the systems students. There were no 
differaxies between systems and traditional classes' performance on 
the tran^xsrt test. 

Chemistry . Systems chemistry classes received 20-item pre- 
and posttests, v^kereas the traditional classes were not given the 
posttest. Both classes were eKMnistered 12-item pre- and posttest 
on reaction rates, the topic of the primary systems module. 
Additional items were administered by the systems teacher to assess 
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further the iaijacx of the systens approach cn acquisition of 
lorwledge of reaction rates. 

Perf onnanoe on the cheadstcy pretest was moderately related to 
posttest scores (r(53) = .28, £ < .05), but not related to scores 
on the reaction rates pre- or posttests. Similarly, students v*io 
scored hi^ier on the reactior. rates pretest did not necessarily 
score better on the posttest (r(53) = .25, g < •05). Chemistry 
posttest scores, however, were related to both reaction rates 
pretest (r(51) = .62, ^ < .001) and posttest (r(57) = .43, ^ < 
.001) . "lliat is, students vfho did better on the reaction rates 
tests also did better on the entire chemistry posttest. 

Analyses attaipted to examine the inpact of course-taking 
sequence on perfonnanoe on the reaction rates tests. Ftur groi^js 
of students were fanned (systems chanistry, 1987-8/systenB biology, 
1986-7; systaoK chemistry, 1987-8/tradiUcnal biology, 1986-7;" 
traditional chemistry, 1987-8/sfystems biology, 1986-7; and 
traditional chemistry, 1987-8/traditional biology, 1986-7) . 
Differences on the pretest were not significant (Taible 10) . 
Hcwever, an ccnparing chemistry treatments indicated that 

systems students perf onned better on the posttest than did those in 
the traditional classes (F(l,139) = 6.30, g < .01). Systems 
stxidents also gained more points, pre-to posttesting, tl^ui did the 
traditional students (F(1,139) = 6.21, g < -01). CCntrast analyses 
indicated that there were no differences on the posttest between 
systems chemistry students v*» hcd taken systems or traditioned. 
biology (T(102) = -.64), nor were there differenoes between 
students who had taken systems biology students last year and were 
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in either traditional or systans dienustry (T(102) = 1.62) this 
year. However, students in the sfystens chemistry-traditional 
biology gxxAjp performed significantly better than those in the 
traditionea chemistry-systenB biology group on the posttest (T(102) 
= 2.30, £ < .01) . These results may be due to the targeted nature 
of the curriculum modules to ^)ecif ic topics within chanistry and 
biology. Gain from pretest to posttest for thes. grxps was not 
significant. Gedn score for the systens-systans group was 
significantly better than for the traditional chanistry-systans 
biology students (T(95) = 2.33, g < .05) . 

BS^i^. The primary measures of content for physics were 39- 
itan pre- and posttests. Students answered 12.34 of the 39 itatts 
c-rrect on the pretest and 20.36 on che posttest, with pretest 
perfonnanoe correlated with scores on the posttest (r(56) = .42, g 
< .001) . No differences were found between students v*ic had taken 
systans and traditional chemistry last year on either the physics — 
pre- or posttests. Both groi?3s of students gained about 7.75 
points from pre- to posttesting. 

Relations hips Among Ability. Achievanent. Oontent. and 
Systans Thirikincr Tests 

Analyse were conducted to examine the relationsh^ among 
ability, content, and systans test performance. First, ^the 
relationship between ability, as measured by the CUT, and the SIT 
was es^lored. Table 11 presents the oorrelations between C3VT and 
sn subscales for studarts in the systems classes. Eterfonuanoe on 




interpretation, translation, and mathenaatics, but also to loops, 
identifications, and systems thinking. Reading and laixfuage skills 
also were related positively to the STI scales, ihe gra^iiing and 
causaliti' scales were least related to CKT performanoe. Overall, 
more able students perfonned better on the STI. These data 
indicate a shift tjxxa last year's results where achievanent on the 
CAT VGxbed. scciles was more proainent. 

Second, analyses thai were conducted to explore influences on 
ocntcnt test performanoe in the sciences. No differences were 
found between GPS classes on the content posttest despite 
•differences in ability. In conparing perforxtanoe on the S&M test, 
oontroUing for ability and pretest performance, systais students 
performed better than traditional studrats. These results indicate 
that there were no overall content performanoe differences between 
the systems and traditional classes. However, on the topical 
material that was taught with the systans approach, students in the 
treabnent condition performed better than those in the control 
classes. 

To explore further differences in posttest perfonnance, an 
ANOOVA vas perfonned, using CKT total as a covariate. Results 
indicated that despite ability differences, posttest performanoe 
differed among the classes. Performanoe on the biology pretest was 
not a significant factor. Differences betweai the sfystens teacher 
and the first traditional teacher were not significant. However, 
tli^ second traditional teacher's classes perfonned better than the 
systans classes on the biology posttest (F(l,101) « 45.04, e < 
.001) . Die ANOCWA indicated that despite ability differences, the 
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second traditional teaciier^s classes perfonnQd better than the 
systene students. 

Examination of the imnunology test and selected iinainology 
items indicated that ^flhen controlling for ability, the traditional 
students still performed better than the systems classes, tto 
differences were found on the transport test vten controlling for 
pretest and ability. 

In chemistry, the systems classes perfomad better than the 
traditional students on the reaction rates posttest (F{1,147) = 
9.13, E < .01). Despite ability and pretest differenoes, the 
treatment effect was sustained in the ANOCWA {F{1,123) = 9.19, g < 
.01) . When the ANOOVA vas repeated, r^lacing the reaction rates 
pretest with the chemistry pretest, the latter was found to not be 
a significant covariate. 

A second set of analyses at±enpted to determine if course- 
taking sequence influenced performance on the reaction rates 
posttest. Stud en ts \iho had systems biology and chemistry averaged 
8.43 points; systems biology-traditional chemistry 8.90 points; 
systems biology-traditJ.onal chemistry 7.25 points; and traditional 
biology and chemistry 7.74 points. Tlv?se differenoes were not 
significant. Hcwever, both CAT score {F{1,86) = 21.18, g < .001) 
and the reaction rates pretest {F{1^86) « 4.55, g < .01) viere 
significant covariates, with course sequence not significant. 

Becaiase there was no co ntr ol class in physics, conpariscns 
were made between students \Ak) had sfystems and traditional 
chemistry during the previous academic year. No differences in 
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perfonnanoe on the physics posttest weane found nor weane CKT scores 
or the physics pretest significant oovarlates. 

A final set of analyses, partial correlations of ability and 
adiievement test scores on the content tests, were performed. 
Both the AIM and CKT total test score were used as s^jarate 
partials (see Tables 12-14) . Controlling for AIM in the GPS 
classes did not substantially alter the correlaticns. Only vAien 
CAT score was partialled out was the relationship between 
perfonnanoe on the STT and GPS posttest H^1m•n l•c>var^ (Table 12) . 
Partialling out AFM made little difference on the biology test 
(Table 13) , vAiereas substantial effects were noted v*ien CAT score 
was controlled. Partialling out CAT dropped most correlations 
substantially, particularly those that involved the STI. Biis 
result is not surprising given the hic^ correlation between STI and 
CAT (r(41) = .58, g < .001) . Ihe correlation between STI and CAT 
score also was hic^ for the chemistry classes (r(39) = .55, g < 
.001) , yet oontrolling for CAT perfonnanoe caused only two 
correlations (chemistry posttest-STI and chemistry posttest- 
reaction rates posttest) to drop substantially (Table 14) . Again, 
controllLng for AFM did not alter the correlations. 

Ooncltisions and Educational Btolications 
Results indicated that the BUHS teachers made substantial 
progress developing and ini)leraenting the systems thinking €^rt>ach 
in their class es . Differences in ^proach and instructionEd 
strategies '«re noted fron Year 1 to Year 2 (Mandinach & Uwrpe, 
1987) . In general, changes that occurred in the science ooui.'ses 
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had positive cutoones. Differencses in the Vlar and Revolution 
saninar had detrimental effects on learning outocnes. 

The results indicated that as the e^proach was integrated iiore 
effectively in Year 2, performance on systems thinking activities 
inproved qualitatively (as reported by the teachers) . Ihe 
improvement was reflective of the more extensive and intensive 
integration of systesns into the science courses. Basic graphing 
skills and understanding functions continued to be problematic, 
partioilarly for GPS students. However, students began to 
understand loops, causality, and the basics of systens thinking. 
Although knowledge of systems thinking conc^Tts inproved, students 
had diff ioilty m odeling. Particularly distressing was the decrease 
in modeling performance shown by the War and Revolution students. 
On the positive side, the GPS and biclojy students inprx>ved 
substantially from Year 1. Oiemistry perfonnance was conparable 
across the two years. _ 

Results fran the con te nt tests were mixed. In GPS, althou^ 
the traditional classes performed better initially and were more 
able, there were no differenc^^ between grtxps by posttesting. 
There were, hcMever, qualitative differroes in the way systens and 
traditional students z^roached problems. Traditional students 
cxxioentrated more on terminology, wiiereas systems students examined 
the oonpofc^ents of behavior over tijoe. In biology, differences 
among the teachers contributed to performance on the postt-est, with 
the systems teacher^s stuteits scaring better than one traditional 
teaciier's classes but worse than the second teacher^s students. 
Overall/ there was no pasti est performanae difference between the 
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systeons and traditicsnal classes. Ihere also were no differences on 
the sm test, the ineasuD2 most closely related to the systens 
modules. However^ the second traditional teacher's classes 
outperformed the systems students on the iimwnology test and 
selected imnunology it^^. 

In contrast^ the systems chemiscry students performed better 
than those in the traditional classics. FXarthermore^ performance 
was 6eper)der± cn vihether students took systems ctenistry rather 
than their biology treatmoit oondition. There were no ocnparison 
grtx^Ds in physics. Hcwever/ students expressed that systew^ was a 
tool they oould a^ply to solve problems in their classes 
(Mandinach, 1988a) . 

Data from the STAd Proj act's two years of research are 
sanev*iat mixed in their ability to provide ooncltisive results about 
the impact of the systems thinking approach on learning. Several 
e)q)lanations for this situation can be rendered: (a) the length of 
time required to develop a curriculum innovation such as the 
systems thinking c^roach and find a^ropriate applications for it; 
(b) the need to integrate the innovation into existing instruction 
so that studCTts are able to make e5q)licit ties between the 
irmcvation and trEditional methods; and (c) the need for 
coordinated data collection that will speak directly to the 
research questions. The results do^ however, provide valuable 
insists into hew such a curriculum innovation taices roots, 
develops, and hfyar m es a fully integrated instructional strat^. 

CXirriculum development made substantia p rogress over the 
project's two years. HcMsver, there is much more work that must be 
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cJone to attain the desired cxxirse sequ^ioe. Tha teachers iincJertc»k 
a massive and labor-intensive project. Curriculum development is 
time consuming and must be done during the sunroer vihen teachers can 
devote blocks of unintem^xted time to wcaSc. It is iiipossible to 
learn a nev body of knowledge and plan instructional materials in 
the 20 to 40 minute blocks of free time given to teachers during 
the school day. Bius it is understandable that developnent of the 
systans thinking approach has emerged slowly. Ihe teachers worked 
extremely hard to identij^ appropriate topics to vihich the approach 
oould be sailed. Ihey planned and inpleraented those modules in 
their classes. Achlevienient then was measured. 

Ihe research paradigm may be appropriate for many studies but 
does not e^ply in the present case. Systens thiiiking is an 
analytic technique or problem solving tool, not a content area 
^ihose acquisition can be assessed by a sisple pretest/posttest 
design. Rather, the systems ^^proach, as an analytic tool is 
expected to influence cognitive prrw?MSfyf=; or the methods by which 
learners abroach tasks. This is xwt to say that systaiB thinking 
will not affect the acquisition of content knowledge. Ihe a^aproach 
indirectly should influence that acquisition throu^ the 
^ jvelopnent of effective hi^^ier-order cognitive processes. Because 
these processes often take substantial time to emerge, it is likely 
that the duration of STACI provided an insufficient test of the 
approadi's potentials. A tnae test of an innovation's irrpact mast 
be longitu din al and make provisions for e^ipropriate ^^w^ and 
resources to sqpport teachers' curriculimi development efforts. 
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A csorresponding issue is how the e^roach is integrated into 
courses, partioalarly sequeixses of ocxirses such as in science. 
Students need to a sense of feel csontinuity and connection within 
and across ocurses. When systems thinking first was inplemented, 
students perceived the approach to be ocnpletely ind^)e«3ent of the 
science contott. There was systems thinking and there was science. 
Systems thinking was not seen as a tool to be applied to the 
content to help students solve problems. To seme degree, the lack 
of connaction escalated during the second year vAien systens 
concepts were reintroduced rather than reinforced. The lesson to 
be learned fixm the BUHS es^jerienoe is that explicit connections 
between the course content and systems thinking as an analytic tool 
nust be built into the instruction. RirthenBore, planned linkages 
betweei courses within a d^artmental sequence need to be developed 
to insure sufficient carry-over without redundancy. Students 
should be told esqaicitly about the generic use of models to solve 
problems and how mnd<»ls will be used in instruction. Furthermore, 
the nature of the systems activities and e5?»ctations for learning 
a new problem solving tool should be meKie e9q)licit. 

From the research perspective, it is inperative to obtain 
ccnparable data from control and eaqerimental classes on targeted 
topics taught with the innovation to assess its inpact. STACI lost 
crucial data in Year 2 that would have provided insic^ts into the 
innovation's iapact. However, as Cronbach (1963) noted, tSe power 
of course evzdua^lon is not found in formal oonparisons of control 
and treatment gt\ .' test data where real differences get lost 
amid a plethora of results. Bather, "it is clearly iirportant to 
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^praise the student's general ecixationed growth, vihich curriculum 
develcpers say is more inportant than mastery of the specific 
lessons presented" (p. 674) . Cronbach reocmnencls small, well 
controlled studies that enable researchers to examine alternate 
versions of the sane course to study and can produce results with 
explanatory value. The three STACI Project substudies (dine, 
1988; Mandinadi, 1988a, 1988b) attenpt to !*^r^<i this need. 

Another issue that emerged was the role of the teacher. The 
inplenientation of the systems thinking e^roach, particularly 
tTecaus e of its theoretical and technological founcJations, requires 
]cncwledge on the part of the teachers. Gaining a worldng knowlec^ 
of system dynamics, the SIEIIA software, and Macintosh is 
substantially different tram acquiring infonnaticn about another 
topic within content areas of expertise. Teachers need to feel 
ccmfortable with their knowledge of the approach so that they -can 
iapart that knowledge to their students. This is not to say that 
teachers must be expe'+s, but rather have working experi&nx with 
the s^roach. Bie STftCI Project observed seme reticence in the 
EUHS teachers. The reluctance occurred because teachers vere not 
ocnpletely confident in their knowledge of the approach. The 
iaplemattation of the approach reflected this lack of conficViiv^e. 

Teacher training is a partied answer to the problem. Althou^ 
all EUHS teachers received in-service training in systens thinking 
and one took a sabbatical to take a university-level systens - 
ocurse, there still were questions of knowledge, confidence, and 
control. Teachers were experienced but not expert. Yet they were 
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reluctant to risk classnxm control by shifting responsibility for 
learning to both students and the teacher. 

Hie systens thinking e^proadi can change the nature of the 
classrocra. The teacher no longer is the sole expert in the 
classrocm, with control of content, knowledge, aid iiistructional 
procedures. Instead, learning henrmps more interactive with shared 
responsibility among teacher and students. Biis means that 
teachers must take risks and alter instructional strategies to 
facilitate rather than sinply iapart learning. Nbt all teachers 
are capable of or willing to accept this evolving role. 

lirplenentation also requires a willingness to ejq)eriment with 
traditional approaches. While all teachers expressed a willingness 
to esqperiment and devoted a substantial amount of ^-imo to 
curriculum develcpnent, there was variation among the teachers in 
hew systans instruction was inplesnented. Banphasis was placed on 
having studarts think about the nature of prcblens by viewing 
interrelationships among factors. Biey were asked to generate 
explanations rather than merely identify, define, or ccn|)ute 
solutions. Differences were evident among teachers in the use of 
SEEUA, the type and ccnplexity of problems presaited, and the 
manner in v*uch assistance was provided. The variation e^jpeared to 
be attributable to a knowledge of systems theory, comfort with 
es^jenmentation, and instructional style. In two 

ragocy svstens 

instruction was iaplemented as a distinct change in the 
organizat:ion and function of the classroom. Students spent 
considerable time developing and experimenting online with models. 
These esqierienoes were structured to engage students actively in 
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the tasks v*iile the teacher served as a facilitator. In another 
case, systems was applied more traditionally. The teacher 
presented systems ocnoepts largely throuc^i class instructicm in 
v*iidi he sought to shift stixJents' thinking thrai^ directed 
questicaiing. A fourth teacher was severely inpeded in his ability 
to iaplement systems instruction due to lack of familiarity with 
STEUA and the Macintosh. 

In general, thcugji, there were significant developments at 
EUHS as a result of the curriculum innovation. There is no doubt 
that the infusion of the systems thinking appcoadi into the science 
and the War and Revolution seminar changed many of the teachers' 
instructional strategies and procedures by viiich to present 
traditional concepts. Given the ongoing nature of the curriculum 
develcpnent effort, it is likely that changes will continue to 
occur as teachers revise and inplement their curriculimi modules. 

The results, however, do indicate that the use of the 
technology of the Macintosh ccnputer and the STEUA software make 
accessible to students and teachers the modeling capabilities 
heretofore found only on powerful mainftame oonputers. Such 
modeling teoadens the range of cognitive r^resentations and 
instructional strategies that students and teachers can bring to 
bear in solving problems. The systems thinking e^roach, 
particularly with the Macintosh's graphics ard mouse technology and 
the STELLA environment, allows students and teachers to develop and 
inplement dynamic models of systems by using a variety of abstract 
rqatesentations to ejqplore and make concrete many phencroena. It is 
our hcpe that the integratd.on of this learning enviroment into 
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cundLoila has the potential to produce students v*io have a greater 
edacity to understand the interrelated and ccnplex nature of many 
phenomena that one owounters in daily life. 
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This researdi was conducted under the allspices of the 
Educational Technology Oentt^r, Harvard Graduate School of Education 
dnd is si?^)orted by the Office of Educational Research and 
Inprovesnent under contract nuaober 400-83-0041. Any opinic»is, 
findings, and conclusions or reocniDerdations eiqpressed in this 
document are those of the authors and do not n ecess arily reflect 
the vieMs of ETC, OERI, or EES. The authors wish to acknowledge 
ajgh Cline, Nancy Benton, lynn Patterson of ETS, Dr. Barbara Bowen 
of Apple Oaifuker, Inc. , Charles Butterfield, David Clarkson, Chris 
O'Bri&i, Larry Ridiioxlson, and the administration and studaits of 
Brattletoro Uhion Hic^i School. 
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Table 1 

STRCr Project Classes. 1986-1988 



Class 



1986 -1987 
Systems Traditioned 



1987-1988 
Systans Traditicral 



GPS 

Biology 
Chanistry 
Physics 
Total 

VlSar & Revoluticn 



3(47) 3(69) 
4(82) 4(94) 
3(63) 3(64) 
Not included 
10(192) 10(227) 
1(8) 



2(41) 3(47) 
4(80) 5(104) 
3(66) 4(93) 
3(65) 

12(252) 12(244) 
1(10) 



Note . Numbers indicate the rmher of classes (and enrollment) 
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Table 2 

War and R evolution Students^ Perfpnoanoe on the Bsvchoroetric 
Instruments; Relevant Oonparisons to Systems Classes (in %) 

Instrument GPS Biology Chemistry Physics W&R 



Figural Analogies 


1986- 


-1987 Courses 






65.10 


69.90 


75.30 




90.00 


Pcogxessive Matxioes 


69.87 


78.73 


85.53 




99.07 


Diag. Relatiorriiips 


46.87 


56.07 


62.07 




86.67 


Letter Sets 


60.60 


71.73 


71.73 




86.67 


Deductive ReasGning 












CAT Meaoing 




70.75 


84.89 




95.86 


CUT Language 




62.00 


82.98 




95.00 


cur Math 




68.04 


81.89 




98.14 


CXL' Total 




66.60 


85.11 




97.5: 


Figural Analogies 


1987- 


■1988 Courses 






57.03 


71.16 


74.92 


72.20 


86.00 


Progressive Matrices 


66.13 


83.09 


86.20 


85.36 


98.00 


Diag. Relationships 


51.48 


67.92 


62.18 


76.00 


87.33 


Letter Sets 


57.84 


66.86 


71.95 


71.22 


77.33 


Deductive Reasoning 


24.26 


39.03 


69.15 


56.31 


63.33 


car Reading 


55.26 


69.12 


76.38 


85.57 


94.90 


CUT language 


45.30 


63.90 


73.92 


83.88 


94.80 


CAT Math 


54.81 


73.73 


76.98 


84.82 


95.10 


CAT Total 


50.74 


69.03 


77.14 


86.78 


96.60 



J**^- aGES'88 = 27. nBio'88 = 67. ajjem'SS = 53. 
I2Etiys'88 - 50. 13m&r'88 " 10- ISSPS'^^ = 46. 
llBio'87 « 75. IJcheni'87 » 54. QfJK/87 « 7. 
Soores for the seven War and Revolixtion students 
oonaxrrently enrolled in phr/sics appear in W&R, not 
physics. 

Scores for the two Mar and Revolution students 
oonojrrently enrolled in chanistry appear in VISR, not 
ohemistzy. 
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Table 3 

Intercorrelations Between Referenoe Batteries, Year 1 and Year 2 



Year 2 



Year 1 



Fig. Anal. Matric.s Diag. Rel. Let. Sets 



Figural Analogies 
Progressive Hatrioes 
Diag. Relaticnsh^ 
Letter Sets 



.20* 



.32 
.47 
.27 
.22 



.30 
.42 
.43 
.34 



.16* 



.30 



.24 
.26 
.46 



.17* 



.23 



Note , n « 204 - 216. * = g < .01. 

E < .001 for all other correlations. 
Mm = 7.26. M3VY2 « 7.11. 
IMMYl = 12.23. JftHff2 = 12.23. 
IPOI = 8.73. MERY2 = 9.88. 
I1L5Y1 = 10.93. MLSY2 = 10.20 
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Table 4 

Interoarre lations Anoncf Ability emd Referenoe Battery Measures 
EA AEM DR IS CRT PEAD lANG 















AFM 


.40 










CR 


.31 


.40 








IS 


.24 


.33 


.41 






CRT 


.24 


.29 


.40 


.29 




READ 


.25 


.36 


.48 


.26 


.36 


lANS 


.23 


.38 


.48 


.36 


.38 


Mm 


.38 


.45 


.52 


.38 


.39 




.32 


.46 


.56 


.38 


.43 



Note. EA = Flg-jral Analogies. AHf = Advancjed Pnagtessive 

Matrices. CR = Diagranning Relationships, is = Letter 
Sets. CRT = Deductive Reasoning Test. READ » CKI leading 
Subscale. lANS = CPU Language Subscale. VKm = CKT 
Mathenatics Subscale. TOTAL = CAT Total Battery Score, 
n = 336 to 38e. E < .001 for all correlations. 
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Table 5 

Reliabilities for the STI Sifcscales 



Subscale Alpte 



Graphing 

Grcifiuiig Interpretation 

Gre^shing Translation 

Graphing Mathanatics 

Loops 

Causality 

Identifications 

S^ ntans Thinking 

Total Test 



5 


.45 


7 


.60 


6 


.62 


3 


.84 


4 


.82 


3 


.94 


33 


.95 


7 


.79 


68 


.95 



Note, n = 214. 
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Table 6 

Interoorrelations amoncr STI Subscales 



GR go: GT GM DDOPS CAUS ID 



GR 










61 


.34 








GT 


.31 


.42 






GM 


.24 


.31 


.42 




LOOPS 


.23 


.48 


.34 


.33 


CNJS 


.31 


.38 


.39 


.18* 


ID 


.26 


.44 


.50 


.38 


ST 


.17* 


.39 


.44 


.30 



Note . GR =ClrE^iiing. GI = Gn^iiuig Interpretation. 

GT = GTc^shing Translation. GM = Gre^iung Hathenatics. 

ia>FS » loops. CMB = CEnisality. ID = Identifications. 

ST - Systens Ihixiking. 

n = 214. * = E < .01. 

e < .001 for 201 other cxarrelations. 
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Tabic 7 

Descrimdve Statistics tor the gn Subscales 



Subscale Maodsum Soore M 



S.D. 



Gre^3hing 

Graphing Interpretation 

Graphing Translation 

Grs^iiing ttathemtics 

Loops 

Causality 

Identifications 

Systems Piihkinc? 

Total Test 



Kbte. n = 214. 



6 


4.98 


1.10 


19 


11.98 


4.10 


17 


8.98 


3. SO 


5 


1.82 


2.04 


17 


10.66 


4.83 


9 


7.82 


2.34 


33 


19.52 


9.59 


34 


13.93 


5.78 


140 


79.70 


24.43 
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Table 8 

CJcpparlsons of Perf ormanoe on the STI (in %) 



/liubGcale 




GPS Biology 


Chenistzv 


Physics 


W&R 




1986-1987 












57 


65 


80 




92 


GrE^liing Interpretation 


35 


52 


69 




98 


Grs$3hing Translation 


34 


47 


57 




91 


Gre^iung Kathonatics 


8 




53 




100 




27 




62 




97 


Causality 


31 


5A 


65 




98 


Identifications 


42 


55 


59 




94 


Systems Thihkincr 
Total 


19 

31 


33 
47 


35 
56 




86 
92 




1987-1988 


courses 








Grs^iiing 


76 


82 


82 


90 


89 


Gre^ahing Interpretation 


47 


63 


65 


70 


66 


Gra^iiing Translation 


36 


49 


55 


66 


68 


Gre^ahing Mathenatics 


4 


26 


42 


64 


82 




39 


73 


60 


67 


74 


Gaoisality 


72 


92 


86 


91 


100 


Identifications 


28 


63 


63 


72 


81 


Systems Thinkincr 
Total 


26 
36 


43 
59 


42 
58 


46 
66 


55 
72 



Note- Q3ES'88 * 34. nBio'88 = 65. |lcheni'88 = 60. 
nEli^'88 " 48. r^£R'88 " 9« i3GEB'87 = 46. 
nBio'87 ■ 70. rjchan'87 = 56. lSffiR/87 = 1, 
Scores for the seven Weu: and Revolution students 
ooncurrently airolled in physics appear in both WiJR and 
phonics. 

Scores for the two V<ar and Revolution students 
concurrently enrolled in both chemistry c^jpear in both 
W£R and chanistxy. 
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Table 9 

Ocxirse Breakdowns for the sn Subseales 



— ^ Biology Chemistry Fhysics War & Rey > 

M S^D^ M M S^D^ M S^D^ M S^f 



GR 
GI 
CT 
Of 
IP 
CS 
ID 
OT 
TT 



4.56 
8.92 
6.15 
.18 
6.65 
6.50 
9.12 
9.03 



1.31 4 
3.80 11, 
3.87 
.58 
3.74 
3.36 
8.59 



8 
1 

12. 
8, 
20, 



5.30 14, 



,91 
98 
29 
32 
45 
25 
83 
72 



1.06 4 
4.30 12 
3.20 
1.99 
4.12 
1.63 
8.89 
5.50 



9 
2 
10 
7 
20 
14 



.93 
.42 
.42 
.08 
.27 
.75 
.67 
.27 



1.19 
3.51 
3.03 
1.97 
4.48 
2.40 
8.18 
4.81 



5.39 
13.30 
11.18 

3.20 
11.37 

8.20 
23.85 
15.57 



.79 5 

.71 12 

.03 11 

.84 4 

.17 12 

.98 9 

,88 26, 

.89 18. 



33 
56 
56 
11 
67 
00 
78 
67 



.50 
3.88 
3.05 
1.69 
6.18 
.00 
4.68 
2.60 



51.09 22.63 82.75 21.67 81.00 19.00 92.07 20.11 100.67 12T83 



Note. GR =Craphing. d = Graphing Interpretation. 

CT = Graphing Translation. Of = Graphing Mathanatics. 
IP = loops. CS = Causality. ID = Identifications. 
ST = Systans thinking. TT = Total test score. 
OSES " 34. xiBio = 65. rjctea = 60- BOv/ = 54. 
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Tsdale 10 

Perfonnanoe on the Chadstrv Reaction Rates Ttest bvr C durse-T^ina 
Sequ^ioe ^ 



M 



S.D. 



n 



Systems Chemistry 
Systems Biology 
Traditional Biology 

Treditional a»nistry 
Systems Biology 
Traditional Biology 



Systems Chemistry 
Systems Biology 
Traditional Biology 

Traditional Chemistry 
Systems Biology 
Traditional Biology 



Systems Chemistry 
Systems Biology 
Traditional Biology 

Traditional Chemistry 
Systems Biology 
Traditi o nal Biology 



Pretest 






3.54 


2.22 


37 


3.05 


1.93 


19 


4.06 


*> A A 


lo 


3.97 


X. 




3.93 






4.00 


X. /o 


J4 


Posttest 






8.66 


2.32 


41 


ft A'^ 


2.69 


21 


8.90 


1.89 


20 


7.55 


2.34 


65 


7.35 


2.40 


31 


7.74 


2.30 


34 


Gedn 






5.U 


3.01 


37 


5.42 


3.24 


19 


4.78 


2.80 


18 


3.72 


2.04 


62 


3.71 


2.22 


28 


3.74 


1.91 


34 
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a-d-ila 11 

Oor telatj.ons Between Adiievanent and Systars Perfomanoe 



Subscale Reading language Mathanatics Itotal 



Graphing .21* .32 .24 .29 

GTE^ihing Interpretation .28 .38 .43 .40 

Gra^iung Translation .42 .47 . 53 .52 

GrsQihing Mathenatics .44 .46 .56 .53 

loops .28 .35 .42 .37 

Causality .18* .23 .26 .24 

Identifications .51 .53 . 53 . 58 

Systans Unnkin5 .42 .43 .51 .50 

Total Test .53 .60 .65 .65 



IJte. * = £ < .01. £ < .001 for al. other cxirrelations. 
JQr = 206. IJl = 206. = 205. nj. = 204. 
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Table 12 

First-ord e r and Partial correlations eanorq GPS and Ability Tksts 







GPS Pre 


GPS Pc3st 


Speed 


Systems 


X 


.45*** 


.58*** 


.63*** 


GPS Pretest 


• 46** 


X 


.43** 


.42** 


(IPS PL^LtiPof" 


• oo 


.45 




AO** 

.48 


Speed 


.70*** 


.44** 


.58*** 


X 


Am 


.46** 


.15 


.44** 


.43** 


systems 


X 


.68*** 


.48* 


.61** 


GPS Pretest 


.70*** 


X 


.42* 


.64*** 


GPS Posttest 


.62*** 


.47* 


X 


.46* 


Spe^ 


.62*** 


.65*** 


.47* 


X 


car 


.47* 


.25 


.60** 


.18 



Note. * « £ < .05. ** = e < .01. ***«£< .001. 

Bam = 31. ijnj^i = 21. 

Zero-<wder cxjrrelatia . 5 are in the lower triangle. 
Partial correlations are in the vpper triangle. 
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Table 13 

Flrst-ort3er and PBa±lal Oorrelations ainoncr Biology and Ability 
Tests 





SystesQS 


Bio Fxe 


Bio Post 


Imnun 


Sel Iran 


Trans 


Systans 


X 


.24 


.55*** 


.64*** 


.64*** 


.42** 


Bio Pretest 


.22 


X 


.19 


.20 


.34* 


.14 


Bio Posttest 


.57*** 


.17 


X 


.59*** 


.49*** 


.49*** 


IrninDlogy 


.65*** 


.19 


.61*** 


X 


.87*** 


.56*** 




65*** 






.88 




.60 


Transpoaot 


if if*** 
• 44 


.11 


.52*** 


.59*** 


.62'*** 


X 


AIM 


.18 


-.07 


.22 


.21 


.19 


.26* 


Systans 


X 


.16 


.29* 


.50*** 


.49*** 


.21 


Bio Pretest 


.27* 


X 


.06 


.19 


.31* 


.13 


Bio Posttest 


.57*** 


.21 


X 


.41** 


.21 


.28* 


Innunity 


.65*** 


.28 


.61*** 


X 


.83*** 


.42** 


Selected Inom. 


.65*** 


.38** 


.51*** 


.88*** 


X 


.47*** 


Transport 


.44*** 


.23 


.53*** 


.57*** 


.62*** 


X 


car 


.58*** 


.24 


.69*** 


.52*** 


.56*** 


.51*** 



Note. 



.01. 



E < .001. 



« e < .05. = £ < 

12mm = 44. " 43. 

Zero-arder oarrelations are in the lower triangle. 
Partial oarrelations are in the vpper triangle. 
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Table 14 

First-order emd Bsa-tial Oorrelations amoncr Chemistry and Ability 
Tests 



Systems Chan Pre Chem B3St RR Pre RR Post 



Systems 


X 


.01 


.39** 


.33* 


.55*** 


Chem Pretest 


.03 


X 


.24 


.21 


-.11 


Chen Posttest 


.40** 


.24 


X 


.64*** 


.48*** 


Reac. Rates Pre 


.38** 


.21 


.64*** 


X 


.21 


Reac. Rates Post 


.56*** 


-.10* 


.49*** 


.23 


X 


K£H 


.35** 


.04 


.U 


.25* 


.13 


Systesns 


X 


-.08 


.29* 


.35* 


.50*** 


Chan Pretest 


.00 


X 


.12 


.08 


-.23 


Chan Rjsttest 


.46*** 


.1€ 


X 


.65*** 


.31* 


Reac. Rates Pre 


.41** 


.11 


.67*** 


X 


23 


Reac. Rates Post 


.60*** 


-.16 


.43** 


.30* 


X 


CAT 


.55*** 


.12 


.44** 


.23 


.39** 



Note. * = E < .05. ** = E < .01. *** = E .001. 
Smm = 47. rjcAT = 41. 

Zero-orcter oorrelations are i the Icj^'er triangle. 
PEa±ial correlaticns are in the i^jper triangle. 
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Appendix A 

Description of Oontert, Refextenoe Battery, and Systems Ttests 



Subject 



Test 



SubGOore Itans Max. 

Score 



GPS 



Biology 



Chemistr 



fhysics 



Pretest 








JUU Year 


Total 


40 


40 


Posttest 








mi Year 


Total 


40 


40 


Speed Nation 


Total 


30 


63 




Sub 1 


5 


5 




Sub 2 


10 


10 




Sub 3 


9 


26 




Sub 4 


1 


2 




Sub 5 


5 


20 


Pretest 








flill Year 


Total 


44 


44 


Posttest 








JUU Year 


Total 


44 


44 




Sub 1 


9 


9 




Sub 2 


14 


14 




Sub 3 


12 


12 




Sub 4 


8 


8 


Transport 


Total 


11 


11 


Innunology 


Total 


31 


31 




Selected 


14 


14 


^Pretest 








mi Year 


Total 


20 


20 


Posttest 








mi Year 


Total 


20 


20 


neacticn Rates 






Pretest 


Total 


12 


12 


Beac±ion Rates 






Posttest 


Total 


12 


12 


Reactions 


Part 1 


25 


25 


Reactions 


Vart 2 


10 


10 


Reactions 


Part 3 


10 


14 


Ihixd Quarter 


Final 


25 


25 


Pretest 








mi Year 


Total 


39 


39 


Posttest 








mi Year 


Total 


39 


39 



Iteachers 
with data 



Sys., Trad. 



Sys., 
Sys., 
Sys., 
Sys., 
Sys., 
Sys., 
Sys., 



Trad. 
Trad. 
Trad. 
Trad. 
Trad. 
Trad. 
Trad. 



Sy.i-, Tl, T2 

Systems 
S^., Tl 
Sys., Tl 
Sys., T2 
Sys.^ TL, T2 
Sys., T? 
Sys., T2 
Sys., T2 



Sys., Trad. 

•Systeacns 

Sys., Trad. 

Sys., Trad. 

Sywtems 

Systems 

Systems 

Systems 

Systems 
Systems 
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^ppeandix A, Ocritinued 



Reference 
Battery 



Systems 
Thinking 



Figural Analogies 


10 


10 


All but 


Advanced Progressive 






Traditicxial 


Matxioes 


15 


15 


GPS 


Diagranming Relationships 


15 


15 




Letter Sets 


15 


15 




Deductive Reasoning 


1 


15 




Total 


68 


140 


Systems GPS, 


Graphing 


5 


6 


Biology, 


GTE^lung Interpretation 


7 


19 


Chesaistry, 


Greqphing Translation 


S 


17 


Ihysics, and 


Math Grs^jhing 


3 


5 


War and 


Toops 


4 


17 


Revolution 


Gusality 


3 


9 




Identifications 


33 


33 




Systons Thinking 


7 


34 





Note. Chemistry Reaction Rates Pr^sttest is a subset of the 
Reactions Vaxt 1 measure. 
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Appendix B 

Pexcent Qsrxect on the Advazxxd Progressive Matrices 





M 


S.D. 


1-2 


.92 


.28 


1— o 


.78 


.41 




.84 


.37 




.83 


.38 


2-0 


.80 


.40 


2-10 


.70 


.46 


2-14 


.0/ 


.47 


2-18 


.59 


.49 


3-4 


.90 


.30 


1-8 


.74 


.44 


1-12 


.66 


.47 


2-4 


.80 


.40 


2-8 


.69 


.46 


2-12 


.72 


.45 


2-16 


.63 


.48 



Note, n = 401. 
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.Appendix C 

Achievanent Test Scxaipes by Teacher and Cdurse 
rin National Percentiles) 



Course 



Treatment 



n 



M 



S.D. 



Reading Subscale 

General Physical Science 

General R^ical Science 

Biology 

Biology 

Biology 

ChaniSs .y 

Chendstry 

Fhysics 

War and Rsvolixtion 

Lanqiaqe Subscal e 

General Riysical Science 

General Hiysical Science 

Biology 

Biology 

Biology 

Chendstry 

Cheodstry 

Riysics 

Nar and Revolution 

Matheaatics Subscale 

General Ihysical Science 

General Riysical Science 

Biology ^ 

Biology 

Biology 

Ctxanistry 

Chanistzy 

R^ics 

Mar and Revolution 

Total Test Score 

General Hiysical Science 

General. Itiysical Science 

Biology 

Biology 

Biology ^ 

Chemistry 

Chemistry 

Ftiysics 

\Jar and Revolution 



Systems 


27 


55.26 




Traditional 


32 


63.84 




Systans 


67 


69.12 




Traditional 


53 


69.38 


19.52 


Traditional 


33 


70.30 


23.07 


Systems 


53 


76.66 




Traditional 


85 


78.36 


X7 . 


£fystems 


58 


86.64 


14 PI 

XH . ^X 


systems 


10 






Systems 


27 


45.30 


PO 46 


Traditional 


32 


60.38 


IQ IQ 

X7 . X7 


Siystems 


67 


63.90 


21.67 


Traditional 


52 


63.85 


18.20 


l^»iitional 


33 


68.70 


22.80 


systems 


53 


74.15 




Traditional 


85 


71.44 


18.61 


systems 


58 


84.97 


15 74 


systems 


10 


94.80 


4 47 


systems 


27 


54.81 


IS PO 

XO . 


Traditional 


32 


68.59 


16 63 

Xw . 


Systems 


66 


73.73 




Traditional 


51 


63.47 


21.09 


Traditional 


33 


70.91 


20.28 


Systems 


53 


77.32 


15.66 


Traditi.onal 


84 


78.37 


15.98 


Systems 


58 


86.12 


12.32 


Systems 


10 


95.10 


3.93 


Sfystems 


27 


50.74 


18.65 


Traditional 


32 


64.03 


17.09 


Systems 


66 


69.03 


20.03 


Traditional 


50 


65. 6G 


17.98 


Traditional 


32 


71.34 


22.26 


Systems 


52 


77.46 


15.11 


Traditional 


83 


77.02 


17.0"^ 


Systems 


58 


87.95 


12.78 


Systems 


10 


96.60 


1.96 



Note. 



Ihe two listings fbr traditional biology reflect the fact 
that there were two teachers in the control treatment. 
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Appendix D 



NAME CLASS 



FIGURAL ANALOGIES 

instructions 

Here are several analogies \n the form of figures rather than 
words. The format is the same. There is some relation between 
forms A and B with C and one of the other 4 options. Your task is 
to determine which of the four figures to the right completes 
the form, A: B :: C: D. Look at the example below The A form is a 
small, solid colored circle. B is a large empty circle. C is a small 
solid colored triangle. Which of the four figures completes the 
pattern? A large, empty triangle or 2 is the correct answer. 
Work through the itemsHCircle the an^ -er that best completes 
the pattern. 



Example: 



51 



□ 

I* 

Q 
□ 




d. 




a. 



U C 



ov 

□ □□ 




^' v C» d. 





O □ D 



Appendix E 

Percent Cbrxect on the Figural Analogies Test 



Itan 




S.D. 


1 


.84 


.37 


3 


.47 


.50 


5 


.83 


.37 


7 


.94 


.23 


9 


.32 


.28 


2 


.90 


.30 


4 


.20 


.40 


6 


.57 


.50 


8 


.68 


.46 


10 


.69 


.46 



Nbte . n = 401. 
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Name 



Appendix F 



DIAGRAMMING RELATIONSHIPS — RL-2 



anljnals 
lions 



iLTlT tt'Vr^ '° Of My cf the circles 



birds 
pecs 




trees 



the le»:JJ'd" ^l'""' of ">'»f • »o- «. to choo« fro. 

correct relatlo^Xfcr! ! ' °' ^" that shows the 

Now try chese sample items: 

O ® O § 0" 



1. Animals, cats dogs 

A B C D E 

2. Desks, furniture, pencils 

A B C D -E 
You should have marked A for 1. and E for 



Of the'^leTo" "co'":c;'c"oTc"s "l^^: T'"."' "IV" ' '"«'°° 

...... .less heee":ri:^:r"„e l5::tn-„!ii-iiL^;Li: ^r^iur 



ERJC ^ '"^•'^ ^'^^'^ ^"^"^11- ASltED TO DO SO 

mmmmm Copyright Q 1975 by Fducat l.v..: r,.srln.. s-rv<.« 41, ....u.. 



You 

n you have finished Part 1, STOP, 
so. 

« ;;5 



.0 



Oo (§ § o@ 



^ B C 

16. Shirts, things made of cotton, blue things 

A B C D E 

17. Pines, trees, stones 

A B C D E 

18. Gems, diamonds, hard things 

A B c D E 

19. Trousers, shirts, hats 

A B C D E 

20. Chairs, liquids, fishing rods 

A B C D E 

21. Dogs, bears, white animals 

A B C D E 

22. Things painted red, cars, bicycles 

A B C D E 

23. Storms, bad weather conditions, hurricanes 

A B C D E 

24. Potatoes, mice, animals 

ABODE 

25. Table ware, objects made of sUver, knives 

A B C D E 

26. Dogs, poodles, animals 

A B C D E 

27. Trees, tomatoes, grains. 

A B C D E 

28. Shades of red, colors, sizes 

A B C D E 

29. Pets, rabbits, white anlaals 

A B C D E 

30. Baskets, hats, objects made of straw 

A B C D E 



Copytt,ht (V) 1975 b. Education., .J?,.. 



^ipendix G 

Peroent Qarrect on the Diagranndng Relationships Ttest 



Item 


M 


c n 


26 


.57 


• DU 


18 


.34 


• 4 / 


20 


.91 


OQ 
• ^7 


22 


.56 




24 


.89 


.31 


26 


.76 


.43 


28 


.76 


.42 


30 


.47 


.50 


17 


.93 


.26 


19 


.62 


.48 


21 


.57 


.50 


23 


.59 


.49 


25 


.41 


.49 


27 


.66 


.47 


29 


.50 


.50 



Note, n = 401. 
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Name 



Appendix H 



LETTER SETS TEST — I-l (Rev.) 

Each problem In this test has five sets of letters with four letters 

In each set. Four of the sets of letters are alike in some way. You 

are to find the rule that makes these four sets alike. The fifth letter 

set is different from then and will not fit this rule. Draw an X through 
the set of letters that is different. 

NOTE: The rules will not be based on the sounds of sets of 
letters, the shapes of letters, or whether letter combinations 
form words or parts of uords. 



Examples ; 

A. NOPQ d)|^ ABCD HIJK UWX 

B. NLIK PLIK QLIX VLIK 

In Example A. four of the sets have letters in alphabetical order. 
An X has therefore been drawn through DEFL. In Example-R. four of the 
sets contain the letter L. Therefore, an X has been drawn through THIK. 

Your score on this test will be the number of problcos marked correctly 
minus a fraction of the number marked incorrectly. Therefore, it will not 
be to your advantage to guess unless you are able to eliminate one or more 
of the letter sets. 

You will be allowed 7 minutes for each of the two parts of this test. 
Each part has 1 page. When you have finished Part 1. STOP. Please do 
not go on to Part 2 until you are asked to do so. 



DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO 



Copyright 0 1962. 1975 by Eaucatlonal Testing Service. All rights reserved 
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ERiC 



Page J 



1-1 







Part 


2 (7 


minutes) 




16. 


ABCX 


EFGX 


IJiCX 


OPQX 


uvwz 


17. 


LNLV 


DTFL 


CLNL 


HRLL 


LLWS 


18. 


ABCE 


EFGI 


IJKM 


OPQT 


UVWY 


19. 


GFFG 


jccd 


STTS 


RQQR 


MLLM 


20. 


DCDD 


HGHil 


MMLM 


QQQR 


WWW 


21. 


FEDC 


^^KJI 


DCBA 


HGFE 


UHG 


22. 


BOBB 


BFDB 


BHBB 


BBJB 


BBLB 


23. 


BDCE 


FHGI 


JLKM 


PRQS 


TVWU 


24. 


BDEF 


FHIJ 


HJKL 


NPQR 


SVWX 


25. 


NABQ 


PEFS 


RIJV 


GOPK 


CUWG 


26. 


PEGF 


KLHJ 


NOQP 


PQSR 


TURS 


27. 


AOUI 


CTZR 


JHTN 


PBRL 


RTVH 


28. 


BEPy 


HJTX 


KNRZ 


KOSV 


WRPM 


29. 


BJIBR 


QQAR 


FTEF 


JXIJ 


SSCS 


30. 


QIFfl 


ecu 


BCOR 


ZR£D 


JIFC 



DO NOT GO BACK TO PART 1 AND 
DO NOT CO ON TO ANY OTHER TEST UNTIL ASKED TO DO SO. 



STOP. 



Copyright © 1962. 1975 by Educational Testing Service. All right* reserved. 
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i^pendix I 
Percent Oorrect on the letter Setr itest 





M 


S.O. 










.92 


.28 




.83 


.38 




.85 


.36 




.90 


.30 


24 


• / / 


.42 


^6 


.54 


.50 


28 


.17 


.38 


30 


.08 


.28 


17 


.86 


.35 


19 


.80 


.40 


21 


.83 


.37 


23 


.76 


.42 


25 


.62 


.4S; 


27 


.43 


.50 


2S 


.31 


.46 



Note, n = 401. 
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Apperdlx J 



mXJCnVE RE'VSONING TASK 



Elliot, Iton, Lois, Hancy, and Ftan, vAvDse last names are Adans, 
Der/is, GortJcn, Harris, and Jackson, have favorite types of books, 
The books are nysteri', biography, scienca fiction^ ocroBdy, 
poetry. 

Match e arything up from the clues belcw. 



1. Jackscn and the person favorite is nystscry bou^t Harris 
his favorite book. 

2. Adaons ^sends aie hour each nic^t reading his favorite, v*uch 
isn't biogcapt^. 

3. Ii5is, Gordon, and the girl uhoee favorite is mystery v/ent to 
the bocik store with Elliot and iVlans. 

4. Hie girl vAwse favorite is "cienoe fiction taUoed to Lois 
about the latest best sellers. _ 

5. Adams and the person \ihoBe favorite is ocoBdy walked to school 
with Lois. 

6. Fran's favarite is not mystery, but she likes it alot, anyhow. 
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MAIRIX KR CECUCnVE REASONING TASK 



Helpful hijic: To fill in the dart, make a note in each box if the 
oa*inatiaj can be eliminated. Wien there is only one blank box 
left in the. rw or oolunn within the category, marie that box as a 
reninder tiiat you have found one of the solutions, ccntinue 
tliitxjgpi the clues, eliminating as- many of the boxes as you can. 





A 


D 


G 


H 


J 


i M 


B 


SF 


C 


P 


E 






















R 






















L 






















N 






















F 






















M 






















B 














SF 














C 














P 








-1 
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Appendix K 



Systesos TtdrHdnj Inetxunent 
Kay, 1988 



Name 



Class 



John was training to run the Bcaston Marathon. To checJc his progress, 
he graphed »*jjs times each week with the following results: 

Week Minutes 
1 185 
5 180 
10 175 
15 188 
20 185 
25 180 
30 170 

1. Circle the set of axes that best describes the results? 

A 190 B 

185 

180 

JON r Mm 180 

175 



185 



175 
170 



"1^ 



1^ 



170 



2V 
JUBEX. 



200 
195 
190 
185 
MIN180 
175 
170 
100 



MIN 



185 
183 
182 
180 
175 



2^ 



2. Now select the apprt^ariate axes and graph John's training times. 



3. Is his tredning working? Why? 
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Rainfall was reaorcled for six months last year. Based on the 
neasurenents, here are the number of inches that fell per month. 



Rainfall 
(Inches) 



10 



8 



E 



? 



4. What are the ooonJinates for point C? 



B 



5. airing what month did it raiii 4 inches? 

6. Hew many inches of rain fell during month 8? 



1 



7. A student prepares for an escamination. The more she prepares, 
the better her perfonnanoe, ip to a point, after viiich 
additionrd study does not iaiiove her test scsare. Translate the 
this re] aticnship into a graph. 
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8. Annunt of exercise causes changes In %«ight. Graph the 

follcwing relationship. As exercise incTBases, weight decreases. 
Make sure to label the appropriate axes. 



When Susan took physical education, her ability to run the mile 
steadily iaproved until she was able to run the distance in 8 adnutes. 
She contiiued at that rate until the end of the school year. When 
S usan returned fran sunner vacation, she found that her speed had 
dropped to 11 miiwtes per mile. Soon after and with some practice, her 
speed quiday returned to 8 mixwte miles. 

9. Gra^ the pattern of Susan's for the mile. 



''4 



10. ivo students had part-time jobs after school. Each staderTt 
deposited a portion of their earnings into a savings aooount. 
The follcwing graphs show the pattern of savings for each student 
frm January through May. 



50 
40 

Money 30 
Deposited 
($) 20 

10 

0 



student 1 



Month 



Student 2 



50 
40 

Msrey 30 
Deposited 
($) 20 

10 

0 




a. Hew would yt3U describe s^-rlent 2's pattern of savings? 



b. Hew does student I's attem of sa\dngs ocnpare %idth studio 2^s^ 



c. Who saved more money in the fron Januazy to MayT 

d. How nuch more money did that student saive? $ 



e. Assume each student continued to save at the same rate^ plot data 
points for June and July en the cfcafts above. 



11. 



a. Greqph the function: y = 3x - 1 



b. Vtiat is the slGpe of 
this line? 



-if 



10 



-10 



lo 
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12. 



Ntite the function for the stralc^ 
line that passes thixsugh tl« oci^in 
and has 2. slope of -1. 



y 



10 



-If 




X 



-10 



13. 

a. Select one of the sets of axes and graph the relationship 
betMBen difficulty of school wcsrlc and amount of stc V« 



b. Briefly describe the relationship between study and 

difficulty of schcsol vjork (how difficulty affects study time and 
how study tine affects level of difficulty) . 



c. Show hew this relationship can be represented in a feedback loop. 
Add tlie + or - to indicate the sane (+) or oppceloe (-) 
direction of relationship. 



ADCunt 
of Study 



Difficulty 
of Scfacal 
Nock 



Difficulty of 
School Wade 



ABDunt of 
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14. [>escribe the lelaticnship between pcactioe and perfonnanoe, 
perfonnanae and aiotivation, and among practice, perfonnanoe, 
and notivaticn. 




Rractioe Perfannaixje 



A. Ocqplete the follGwing sentences to describe a caiisad relationship. 

B. Then drav an arrov diagram to represent causation. 

C. Next, iniicate if ttie chancre in the second variable is the same (+) 
or opposite (•) ftan the first variable, 

BONFIE: Gdnsmpticn of desserts causes changes in weictfit . 

ODr-^4i]|ption of desserts ^ %ieig^ 

15. Amount of exercise causes ^ . 

exercise 

16. NUnter of births causes . 

nunber of births 

17. The aijnunt of stuJy time causes 

study time 
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Frustxation 




Perfomanoe 



Describe the relationship between frustration and perfcnnanoe. 



19. 



Tai{)erature 




Use of Air 
Ccnditicner 



Descaribe the relationship between taaperature and use of an air 
ocnditioner. 




Tiredness Tiredness Activity 



Describe these two feedback loops. 
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21. For each set of three variables, state vtether it is a stock . 
(level) or a flov (Inflow or outflow) (rate) • Give the ijnit of 
neasure for each variable. 



Exanple : Variables 



Stodc or Flow 
Stock 



Ohit of MaasuTB 
Miober of pecple 



Population 
Birtte 
Doatte 



Flow (Ihflov) 
Flov (Outflov) 



Babies bom per year 
Deaths per year 



a. Ckedits 
Bank BaLLanoe 
Debits 

b. Eva^xsraticn 
Rainfall 
Water 

c. Otxpletions 
Assignments 
Schoolwoiic 

d. Inventory 
Sales 
Production 



22. Par the following set of variables for fli^t patterns, create a 
model using a flow diagram. Briefly a)?)laln how the rnrA^\ works, 

EXntOEz Population, Deaths, Birtte 



Birtis 



Deaths 



> 



Pcpulation 



o 



o 



FLTGHT PAmgN VRRIABIES 

Departures 

Planes 

Arrivals 
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23, Using the flight pattern nodel above, ^t variable(s) might 
affect the luober of departures? 



24. Here is a nodel of a student's recxaxl oollecticn. label the 
runbered parts of this model as a flow, stock, converter, or 
connector. 



Raoord 
Oollecticn 



Cost of 
Ifew Racords 



Alloiianoe 



label the folloving: 

a. b. 

d. e. 

9- h. 




Recxsrds 
Discarded 




Latest Hits 



c. 
£. 
i. 



25. Aocxardim to the nodel , vjhat influenoes the amc«nt of records 
in the oollection? 



26. Aooordim to the nodel . v*at influenoes the nunber of purxiiases? 



27, Hew mic^ the purchase of a ocnpact disc player etffect the nodel? 
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Each year the bluekdn^ed finch oust sake the journey frtn South 
America to Iceland in time for mating seascn. Ohe birds leave 
South Ambrica and step only onoe (in Maryland) to restock their 
food si^ly. ihey must rely on sand crabs as their only food 
source. After garging thanselves, those birds that are able to 
fly (usually a eanall percentage are unable to make the trip) 
continue cn to Iceland^ vtiere they then mate and eventually 
return to South America. Each finch couple »^TT3duc3es two 
babies per year. 



Wiat variables miq^ affect the birth rate of the bluewiraed 
finch? 



Draw a flew diagram to show the mating system of the bluewincred 
finch. ^ 



Daring the fifth year of migration, a disease infected the 
population of creite, cai?sing a severe food shortsige. Ihe 
number of crabs slowly returned to normal by the tenth year. 
!niis ten year cycle recurred. Draw a grqph to shew vtet would 
v^l^en to the finch population as their food source increases 
eid decreases. 
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I ceo 



M.tximua 
Score 



ITg M STATISTICS FOR TH£ ^T^TKHS THIMKTWG TMSTWITH l^WT 
GPS Bfology Cheaistry Physics 



War & Rev. 
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i^Y^endix M 
Pr^>ared by M. E. Ihorpe 
Curricalum and Instnjction 
Ttie science teachers integrated the systerts tninkirn approach 
into instruction in di:=tdnct ways. Becaii^e of variation in Ihcm tiie 
systems thinking approach was integrated into instrv tion, it is 
useful to examine each subject separately. For all content areas, 
the objectives and ger*eral e^roaches to instruction are presente , 
folxcwed by a discussion of the content and methods used in 
teadiing with a systans per:^)ective. Instmctiorad activities^ 
requisite cognitive skills, and content knowledge are described. 
Ocnpariscais between traditional ar.d systans instruction focus on 
information related to topics taix^t with the systeans approach. 
General Fhysical Scienoe (G£S) 

Systems instruction was Ui^ to ttach speed, motion, velocity, 
euri acceleration, oono^Tt^ traditionally difficult to master. 
Althcu^ students learn forraulas and terminology, f&f have an 
intuitive understanding of relationshipc uncJerlying these ideas. 
To promote cct^prdiension, tne teacher used an inquiry-oriented 
format and a c jKbination of traditional and systois methods that 
was si^plemented with structural models. 

Sequence and organization of systems instruction . Systene 
initially was tau^t through exploration of models and their use to 
solve pylons. Drawincr .41 exanples fixro the video "Search for 
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Solutions" and ac±ual eyperienoes, stxidents described problems and 
identified elements of models. To reinforce the notion of 
causality, students were exposed briefly to causal lopt. diagrams 
and asked to construct loops of scientific and non-scientific 
problems. Structured diagrcunming was introduced, in cc. junction 
with the Macintosh, as another roodexing tool. 

Instruction began with a dearaonstrcitiai of a water tank mcxSel 
U5;ing fflELEA. Students worked online, in pairs to build and explore 
struccural models using woricsheets pr^>arBd by Tpchnical Education 
Researcii Centers (TERC) . Students experienced diffictlty with the 
task procedurally and declaratively. Althouc^ directicMis were 
given to guide students throug^i a sequence of stepg>, the objective 
of the activity was not stated, leaving many students loiKsertain 
about what they were doing and Why. AiiOther problem \*oiS the lack 
of famliarity wJth the Macintoshs and STELLA, conpounded by 
operational medicinics and the need to alternate among different 
representations. 

The teacher subsequently adopted a more direct approach. Ife 
presented a sinpler offline model for discussion with the whole 
class. Students were shown how the oono^ of rates was 
represented in a strxictural model. StucJents then worted throu^ 
problems together, arreting graphs of expected rates and flows. 

After a month's interim, systems thinking was reintroduoei to 
teach speerl, distance, aooelpraticn, and velocity, with 
augnentation by traditional methods. Student- -^pent a week on 
offline problems using fomulas. ihey also were required to 
describe the motion of objects in veriaai or gr^dcal form. 
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students then reviewed STFTJA symbols, discussed a inodel of 
acsoeleration, velocity and distance and worked throucfi prxsblems of 
stocks and flews. They worked cailine for one week creating 
structural models. Following detailed instructions they built 
siatple diagrams of velocity (a rate) and distance (a stock) . They 
ran models, changed parameters, and explained results. Th^ also 
used their models to solve traditional word prcblems. 

A more ourpiex model vs introduoed to illustrate relatic»^-3 
esaang acoeleration, velocity, and distance. Students were given a 
stnxrtured diagram and parameters for eaich variable and were 
directed to create graphs, change values, predict, and describe the 
observea .^havior. They were able to do only a few prdDljeros and 
mctfty had difficulty generating and interpreting the grs^iis. An 
aoceleraticn lab then was conducted. Students watched a 
demonstration, collected data, and grs^shed their results. The 
class then ocopared the STEL'A models with other r^resentationr of 
speed and motion. 

Assessment of systans instruction . Speed and motion usually 
are diffiaalt for students to If^am due to the concept of rates. 
The systems abroach presents rates structurally and grs^iiically, 
allowing students to define rates and relationships in a model. 
Additionally, students can create structural diagrans and 
illustrate hew the underlying bAavior changes over time. When 
stxidents are Instructed with tr<iditional mbthods- they of^en fail 
CO grasp tliat rates change over time. They focus on mastering 
formulas. Although they mic^t oncpute the correct answer, they 
often cannot e)q)lain or predict tiie bdiavior ever time. 
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While systans instnjction helped sane students, others had 
difficullY imderstanding speed and motion. Understanding 
aooeleratLcn as a rate of cJiange of a rate of change was difficult 
evcai with systans. Sane students had trouble with the lo^^c 
underlying velocity's r^reser'-atiai as both a stock and a flow. 
Furtheniore, many students initially had difficulty with the 
mechanics of STEUA um il the teacher provided <25^1icit directions 
and sanple models. Ihe teacher expressed dis^jpointment with 
students' explanations of observed bdiavior, despite atypical task 
deanands. Increased exposure to this type of task, as well as m^re 
exanples of process analysis nay facilitate perfonnanoe. 

Oopparison with traditional instruction . The systens and 
traditional teachers wanted students to attain a basic 
vmderstanding of speed and motion. Although they shared this 
goal, they differed in their a^roaches to instructiOT and the 
amount of time devoted to the topic. The ttaditicnal teacher 
dsvoted six weeks to the unit; the systems teacher spent four 
weeks. The traditional teacher osmphasized definitions, 
measurement, and formulas. She closely followed the text, 
enphasizing a ocrpitational approach to speed and motion problems. 
Students were instructed to write formulas, enter the nunfcers into 
the formulas, make conversions, and check their answers for 
accuracy. In caitrast, the systems teacher stressed coxsepts 
underlyirg the fonmilas. Although students were ea^xjsed to the 
formula method, instruction was pi:esented primarily fran a systene 
perqjective. 
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Biology 

In biology, systems was used to present an integrated view of 
biologicad processes. This integrated approach cxaitrasts with 
traditional instruction vAiich typically enphasizes the functicxis 
and chc»racteristics of biologiced processes in a more segmented 
way. In ocnparisoii to otrier sciences, systems instrur ion in 
biology was used prinarily to reinforce ancVor synthesize 
information presented though other approacbcs. 

Systems was applied differently in the instruction of celluGLar 
tran^rt and cellular rerponse to infection. With transport, 
systems was used to reinforce concepts students had learned throu^ 
other activities. Ihey vrorked on several labs related to cell^dar 
tran^rt prior to the cystars module. Ihe systems iftodel was a 
siraulaticn of their lab e}q)eriences. Systems was used to 
synthesize and extend information previously tau^t in inraunology. 
Althouc^ students j^jent several weeks studying characteristics, 
functions, and selected relationships among the cuiponents of the 
inuune system, they had not <iiscussed the way these con{x»ients 
operate as a ^stem. The imoctel was the first time studtents were 
exposed to how these conponents interact during cellular re^xaise 
to infection. 

Sequence and organizaticai of sygfaans instructicai . The s y s t atts 
teacher used a direct approach to instruction. In conparison to 
the other science classes, more time was devc ted to whole class 
presentations and discussion. He spent ^proudmately one week 
xntrod'Kring students to modelinj and STEEIA. As in the other 
sciences, studerits were informed that models could be used as a 
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problem solving tool in many subject areas. Alttuxig^i causal models 
WFire introduced, greater emphasis was placed on teaching systene 
fran a stiucturzd perqjective. 

Students were introduced to structursd diagrans and STEIIA 
thrcuc^ deanonstration and discussion of a popularlon mcdel. ihey 
examined the oaoponents of the model and hew factors affected 
b^vior over tiaae. They also made predictions about patterns of 
beiiavior and altejred them based on changes in the nodel. Grsjiis 
were examined with respect to predictions. To deroonstrate STELEA's 
siaulation edacity, the next model was done in conjunction with a 
lab on the cooling rate of liquid. At fixed intervals, the teacher 
repeated the tenperatures of three soop containers of equal volume 
that differed in initial tenperature. Students reoortaed data and 
created graphc of tenperature changes over time. The teacher then 
demonstrated a structural model of the e3q)eriment. Students 
ocnpared their graphs to the one generated by SIEtlA. As a class 
they suggested modifications to the paramtiters of the model and 
observed changes in cooling rate. 

Students were instructed to o^^xate the ocnputers throu^ the 
software program, Guided Tbur of the Macintosh. After conpleting 
this program students, worked online in pairs to €3q)lore a teacher- 
constructed model of oxygen production during photosynthes^. ihe 
model illustrated the relationship between ligt;t intensity and the 
rate of cocycfen proJuction. Althr:i^ tha teadier had intrtxJuced the 
model pnevicusly in class discussion, students had not sti^ed this 
topic before, and therefore, they had difficulty understanding 
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relationships. Bie use of STELLA facilitated students' subsequent 
perfonnancse in modeling and understanding of the cxyncept^ 

Follcwdjxf a series of learning activities on the topic of 
diffusion and osmosis^ a structural model was introduced to 
reinforce thjsse processes. In this instance^ the teacher led the 
class to construct a occ?.lex structural model containing two 
interrelated subsystems^ one for osanosis and another for diffusion. 
The students and teacher indicated that the process of grxxp 
modeling was helpful to undei^stand the relationships in the model. 
Students then manipulated this model online. Exercises guidad 
students to examine and predict the bdiavior of factors over time 
as well as the relationships amonnr factcars in the modal. The 
module was suranarized by revietdng the SIXUA graflis. 

After a four month interval, students were reintroduced to 
systems instruction through causal loop diagraznaing, beginning with 
single loop structures and progressing to increasingly conplex 
nultiple-loop diagrams. Ihese lessons were d^ in preparation for 
caused modeling during discussion of the iztnune system. The 
teacher devoted a montii to the imnune system. He reviewed 
characteristics and functions of cells and proteixs, hew they 
conbat infection, and different types of imnunity throu^ 
traditional instnjctional e^roaches. Ii^strixtion enfiiasized the 
causes of disease, hew disease is ^read, and the boc^ defeite 
against infection. Students learned techniques to isolate and 
stucty the characteristics of bacteria from lab activities. They 
also examined the effects of different substances on controlling 
the qaresKi of microbes. 
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Following this substantive preparation, the teacher guided the 
class to oonstruct a caused loop diaigram of cellular responsG to 
infection as a means of integrating and extending knowleige of the 
inmune system. The m odel was ocaistnx±ed in successive st^Ds as 
students were prcnpted to suggest and defend connections among 
tactors. Stajdents made predictions about the behavior of the 
inraine respcnse and the effects of new connections on various parts 
of the system. The model was the first time students were exposed 
to a systemic view of cellular re^xsnse to infection. 

Directly related to the imnune system was the model of AIDS 
yJtticti the MIT consultant presenb^d as a catisal loop. The 
consultant Presented the model and cvxJix±ed a class discussicHi on 
ti^ spread of AEDS^ the model's stnxAure and ccnpcnents, and 
policy decisidts related to the disease. 

Assesanent of svst.ems instr\x±ion . Observations and anecdotal 
reports indicated e hi^ lev-el of student engeigement during systeais 
instnjction. Group modeling as an instructional vehicle challenged 
and ^igaged students to participate in these discussions. During 
class discu5>sicns students were able to identify important factors 
within biological systems, suggesc anc: e5q)lain catise-and-effect 
relaticnshj^, and predict aTtccmes. However, results from 
tran^»rt and imunology tests were dise^^pointing. Stulcnts 
indicated that they were confused by certain questions and that 
they needed additional instruction in certain arsas particularly 
grEqphing. 

OcBparison with traditional instruction . Although the systems 
and traditional teachers used the same text, they differed In the 
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amount of time devoted to topics. Ihe systaiB teacher sp&± five 
weeks on both transport and inraanoloc.(y; the traditicxial teauiers 
devoted three weeks of instruction to each topic. Inere also were 
differences in the number and type ot activities used to stqplement 
the text. Differences in ped gogical c^roach were observed 
although the three taachers used class discussion as the primary 
method of instruction. One traditional teacher proceeded slowly 
and had difficulty with classroom management. Ihe other 
traditional teacher used veirious strategies to proraote student 
understanding and retention of information, ahrtxi^ questioninrj, 
visual cues, and analogies, she directed students to connect prior 
knowledge «^ new information. Her presentations and discussions 
were clear, well organized, and fast-paced. Ihe systens teacher 
shared some characteristics of both traditicHrU. teachers. His 
lessons were organized and he used visual materials effo-cively. He 
tended to elicit students' ideas rather lhan to s^ly zesporses. 
His class was attentive although the pace of his lessf«s was slower 
arvA more similar to tiie first traditional teacher. 
Chemistry 

The sfystems aqpproach was used to augment instrix^icxi on 
reactions and providft an analytic tool to prcroote thinking ahout 
cause-end-^ffect relationships and bdiavior change. Although 
reaction rates is not a difficult topic, the teacher anticipdted 
that systems mic^it broaden understanding and provide c. problem 
solving tool that ouuld be supplied to this and other chemical 
ozicepts. Ihe selection of reaction rates as a systems nodule 
represented an ideological shift on the part of the teacher. His 
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initial goal, to examine diemically related social pixbleais (e.g. 
pollution) , proved unsatisfactory because the cfcemistry underlying 
the ncdels was too ocnplex. FUrtheniore, the social problens were 
removed fmn the core rurriculum. Ohe teacher therefore souc^t an 
alternate e^proadi tx> integrate systems instniction into existing 
ocurse content. Hc^ever, identifying appropriate instnjctiOTial 
topics was difficult. Instead he modified instructional activities 
developed last year and focused systans instruction on rates of 
chemical reactions. 

Sequence and organization of systeros instruction . Students 
were introduced to systems throuc^ causal loop diagrans because 
models ocxild be constructed from verbal descriptions without the 
oonplesdty of quantification. Students described patterns of 
bdiavior over time and examined oonnpctions among variables, 
cause-and-effect relationships, interactions, and feedback. 

Three models were introduced: the rate at v*iich dou^ rises; 
reactants and products of the nitrogen cycle; and burnout. The 
roal was to model these processes and describe the relationships 
among factors. DGmGnstratj.ons, guided discussion, and assigninents 
were used to help students construct causal models. 

Ihe topic of reaction rates was introduced with traditional 
methods. Students learned to apply fonnulas to express ? id ccnpare 
reaction rates. Although systems models had not yet been 
oonstru±e*2, students wxe esqxssed to concepts related to sy<;tem 
instriction. They oonuucted es^Deriments and disais.*=;pd the effects 
of alterations in factors such as tenperature and concentration on 
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reaction rates, Biey also constructed graphs of lab results to 
view changes in reactants and products over time. 

IXiring the second week of instruction the teacher gave a 
demonstration of modeling a reaction with STEIIA. Students then 
esqplored STEEIA following a prq>ared script to ccxistruct a chemical 
reacticHi model with predefined connections. This lesson provided 
guided exploration of the operation of the Macintosh and STEUA's 
structural and graphical features. Stuctoits worked online for two 
weriks dc-/elcping and altering reaction models. Models were based 
on previous labs^ enabling students to apply prior knowledge to 
develop hypotheses of eyptxted outooraBS. Online exercises required 
students to construct, alter, and test models with different 
parameters, then predict, describe, and explain the behavior. A 
f insd exercise requii^d students to construct models tc see if they 
oculd apply ij&nsric models to novel problems outside of chemistry. 

Assessment of systaros instruction ^ The teacher expressed 
mixed reactions to systems instruction. He felt he ^jent too much 
time on reaction rates at the expense of other topics, e^)ecially 
given that the concept is not critical for hic^ school chooistry. 
The topic of rates was selected because it was amenable to the 
approach, and therefore offered an opportunity to illustrate how 
systems could be allied to instruction in chemistry. However, 
instruction e xcee d e d conventional coverage of this topic due to the 
time was needed to provide both traditiuial and systens activities, 
particularly student modeling. 

Learning STEIIA was difficult for some students. Th^ had 
prdbleas understanding how to specify and operate their models both 



iriBchanically and logically. The reaction rate fontula was easier 
to apply the operation of STEIIA. On the positive side, the 
teacher indicated that oonstruction of reaction rate nodels 
facilitated students' understanding of chemical equilibrium, a 
traditicrally ocnplex concept. Students understood equilibrium 
more readily than they had in the past. Forthennore, understanding 
equilibrium assisted students to learn topics sudi as acids and 
bases and solubility of prcducts. 

STEUA was generally perceived as a useful si^plenient to lab 
activities. The teacher indicated that although labs were helpful 
to develop conoeptual understanding, students gmioi-iinoc^ ^ere 
confused by the procedu r es and their efforts to find the ri^t 
answer. Moreover, labs did not always work due to inprecise 
met^uds. Ihus, the potential benefits of labs were reduced v*ien 
students did not see the desired outcomes, SIEIIA could r^licate 
observed lab experiences and simulate other conditions without need 
to repeat the e^qDeriment. 

Oonparison with traditiaial instruction . Both the systene and 
traditional students covered many of the same chemistry concepts 
with different irstrtx±ional activities, enphases, and exposure. 
Traditioned classes enphasized mastery of algorithns in contrast to 
developing m odel s to define relationships between concepts. Under 
t^ical conditions, the systems and ti-editional teacAers had 
similar instructional styl-^. Instruction was moderately paced and 
marked by frequent questioning. Etoth teachers used a lecture- 
discussion format and connunicated in clear, concise terns. Hiey 
also used the similau: lab activities. For the study of reaction 
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rates, the systans teacher departed from his usual approach. 
InsteaKl of a using direc± instruction, he organized student 
activities to prcnote inquiry. Students worked online for two 
weeks. Ihe teadier served as a facilitator rather than an 
instructor, re^Donding to questions and providing help as needed. 
Physics 

Systems thinking was used to augment several physics units, 
including notion, xncnentum, energy, and electricity. Ihe nodules 
esqxDsed students to both treKiitional fomulas and the systems 
approach. SIEUA was used to develop a deeper understanding of 
relationships anong physical concepts and observe changes in 
behavior over tine. 

Sequenoe and organization of s-rsteas instruction . Parallels 
between instruction in physics and GPS were found, particularly the 
enftasis on stn^ctural nodeling. Although causal loops were 
introduced briefly, structural diagrasndng was the prinary problem 
solving tool. smiA was introduced with a model of water tank 
drainage. The teacher demonstrated how SIEUA operated vAiile 
discussing the physical relationships displayed in the model. 
Students then spent twD w^eks building structural models of motion. 
Ihe introduction of smiA was facilitated by a nunter of suxients 
vAio r emembered its mechanics ftan systems chemistry. 

Textbocdc exercises that required mathematical formulas, 
providi^ opportunities to solve problems and oonpare traditional 
and systems solutions. I^picadly students woriced on these probleoits 
for several days then discu's:^ the results. SIEUA was used to 
model problems illustrating the conservation of mcmentum. Students 
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T^ere introduced to a general ma jqI of ncmentum and then given 
cnline problems. Using guided inquiry, the teacher helped students 
to focus on important elements in the problem and appropriate 
soluti(» processes. 

Students spent three days building ;i coiplex model of rocket 
propulsion. Ihe teacher reported that students werx^ able to 
construct the modal with varying degrees of prcnpting and 
understood the numerous factors. At the end of the year, the 
teacher returned to the water tank problem. Students conducted a 
lab, collected data, drew grs^iis to Illustrate the bdiavior 
observed, and then developed stnx±ural models. Ihe teacher 
sumarized their eaqserienoe, reviewed the logic of the model, and 
discussed other ways they mic^t have solved the problem. 

Assesfront of systems instruction . The teacher was more 
satisfied with systems instmcticn in physics than GPS. Hiysics 
students were better able to cope with the applications and 
meciianics of modeling. Nevertheless, seme of the same difficulties 
found in GPS were noted in physics. Students differed in their 
need for help and tolerance for ambiguity '/ith instruction throu^ 
guided inquiry. Seme students had problems with the operation of 
STEIIA. Ocnmon problems involved defining gra^iis and initi2dizing 
values. Seme physics stiidents had trouble conceptu2dizing problems 
as diagrams, confusing stocks and flows. Others did not approach 
problems analyticedly, failing to use knowledge of pt^ics to 
predict yihat their structural diagr?^ should look like. As a 
ccxisequance, some students had difficulty interpreting graphs and 
alterinc^ their model based on their understanding of the phenomena. 
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The teacher noted that vhen questioned individually to focus cxi key 
elements of the problem, students reqyxided appropriately. More 
exposure to generic models mic^t be an effective instruction 
tool. Moreover, sane students mic^t benefit from explicit 
instruction on problem solving strategies inclixiing self-regulation 
techniques to direct attention to inportant problem elements. In 
contrast, a larcfe proportion of students vmderstood hew to use 
STEHA. Encouraged by the experience, the physics teacher will 
e3q>and his use of the systems thinking approach next year. 
Sunmary 

Objectives of instruction . Systems thinking was used to 
facilitate understanding of the bdiavior of scientific phencroena 
over time. Teachers eaqnressed dissatisfaction with current 
instructional methods, viewing traditional s^Dproaches as developing 
fragmented knowledge. Ihe approach was used to pranote integrated 
knowledge and develop analytic skills. Althouc^i the jjnwpdiate goal 
was to iiDprcfve knowledge aoquisiticn, the teachers eaqpected that 
skills developed by 'iie systems approach mic^t generalize to other 
disciplines. Thus, systems was used to augment instruction and 
teach heretofore difficult topics and concepts. 

Approaches to systems instruction . Although the teachers 
shared goals and concerns, they differed in their approaches to 
2^stems instruction. C3ontent differences and objectives determined 
hew the.systems e^roach was applied in the fou^' courses. For 
exanple, systems was lased in biology to synthesize and 2^1y prior 
knowledge to broader problems. In con tr ast, the goal in pt^ics 
was to d epar t from the traditional mathematical approach, reduce 
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the en|3hasis on mastery of ooniriitaticjnal foaajlas, and pracote 
understanding and a^splication of concepts underlying these 
fomulas. The systems thinking approach^ used as an anedytic 
problem solving tool^ was seen as a way of achieving that goeil- 

Pedagogical style and the method by >Mch the systems e^roach 
was applied determined how instruction was organized. In biology, 
direct instruction was used to teach causal and stnx±urEd irodels. 
Modeling was taught through teacher-led discussions, si^pleanented 
by online exercises to explore and manipulate teacher-constructed 
models. The other science teachers used inquiry methods to augment 
structural m od e ling. Occputers were used as an instructionad tool 
on vAiich s tu d en ts constructed and manipulated structural models. 

Time constrednts for instruction and curriculum development 
continued to be a concern for the systems teachers. last year, 
teaciiers indicated that too much time was spent on systens modules, 
at the expense of other topic areas. Bie chemistry and GPS/physics 
teachers co n sequently shortened tha^x systems units. Tteachers 
needed to attain a bedance between implementing the systens 
2f3proacii and meeting extemcdly ijposed achievement standards. 
Time cdso was a factor in curriculum development. The demands of 
teaching in conjunction with development of new curricular 
approaches and materials placed heavy dpmands on teachers' time. 

Effects on teacfain? and leaminci . Teachers reported that the 
systems e^procKii heightened their awareness of instructional 
content and gocds. It adso provided new op portu nities for ongoing 
student assessment, particularly because STEUA produces 
representations of the processes and products of learning. STEUA 
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makes it possible to diagnose conoeptual deficiencies and strengths 
firom the cognitive r^resentaticns built in the enviroTment. 

The teadiers reported that systems is a prcndsing tool because 
it r^resents a unique approach to science instruction. Of 
particular iitportanoe is that the approach deenphasizes facts and 
algorithms, focusing instead on the proces3 of understanding hov 
things work. By constricting and manipulating models, students 
develop an intuitive understanding of the systemic and dynamic 
nature of phenorjena. 

SystaocG thinking is an s^roach to teaching and learning 
activities that is at^ical in its eqphasis on process rather than 
only the products of instruction. Ihe focus on solution process is 
likely to be an unfamiliar and antithetical approach, given 
pressure for achievement. It is critical that teachers coniiunicate 
this enphasis to their students and stnicture instructioi to 
reinforoe the pedagogical perspective. Seme students, however, 
will need more guidance to deal with the sqpproac^ because they are 
accustomed to selecting rather than generating correct answers. 

The systems approach promotes active participaticxi from 
students and teachers. Teaciiers demonstrate the processes and 
logic that: underlie model building as an analytic tool. They 
siDQUlate cognitive approaches to problem solving and demonstrate 
modeling as an iterative process. Hypothesize-test-refonrulate 
se q ue n ces foroe students to refine representations of systemic 
phenomena. Thus, the iterative nature of modeling prcvi as 
feedback that is lacking in nixli traditional instruction. 
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Bpleaoientation also requires a villingfness to experiment with 
traditional approadyss. Vftiile all teachers expressed a willingness 
to ejqjeriment and devoted a substantial amonnt of time to 
curriculum development, there was variaticxi among the teadiers in 
hew systeons instructicxi was iirplanented. QifDhasis was placed on 
having students thiiik about the nature of problems by viewing 
interrelationships among factoid. Ihey were asked to generate 
e3q)lanations rather than merely identify, define, or ocnpute 
solutions. Differences were evident among teachers in the use of 
STEUA, the type and conplexity of problems presented, and the 
manner in vftiich assistance wa3 provided. The variation appeared to 
be attributable to a knowledge of systems theory, comfort with 
e3q)erimentaticn, and instzuctional style. In two cases systens 
instmcticn was iiipleniented as a distinct change in the 
organization and function of the classrcxm. Students q)ent 
considerable time developing and e3q)erimeating online with models. 
These experiences were structured to engage students actively in 
the tasks vihile the teacher served as a facilitator, thus, 
representiny a departure from traditional student-teacher roles. 
In another case, systems was e^plied more traditionally. The 
teacher presented systems concq>ts largely through class 
instnK±ion in vdiich he sought to shift students' thinking through 
directed questioning. A fourth teacher was severely impeded in his 
ability to inplement systems instruction due to lack of familiarity 
with STEIIA and the Macintosh. 
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